Abstract
This paper introduces a neural network to solve the structure-from-motion(SfM) problem via feature bundle adjustment (BA), which explicitly enforcesmulti-view geometry constraints in the form of feature reprojection error. Thewhole pipeline is differentiable, so that the network can learn suitablefeature representations that make the BA problem more trackable. Furthermore,this work introduces a novel depth parameterization to recover dense per-pixeldepth. The network first generates some bases depth maps according to the inputimage, and optimizes the final depth as a linear combination of these bases viafeature BA. The bases depth map generator is also learned via end-to-endtraining. The whole system nicely combines domain knowledge (i.e. hard-codedmulti-view geometry constraints) and machine learning (i.e. feature learningand basis depth map generator learning) to address the challenging SfM problem.Experiments on large scale real data prove the success of the proposed method.