Abstract
We present a system for keyframe-based dense camera tracking and depth mapestimation that is entirely learned. For tracking, we estimate small poseincrements between the current camera image and a synthetic viewpoint. Thissignificantly simplifies the learning problem and alleviates the dataset biasfor camera motions. Further, we show that generating a large number of posehypotheses leads to more accurate predictions. For mapping, we accumulateinformation in a cost volume centered at the current depth estimate. Themapping network then combines the cost volume and the keyframe image to updatethe depth prediction, thereby effectively making use of depth measurements andimage-based priors. Our approach yields state-of-the-art results with fewimages and is robust with respect to noisy camera poses. We demonstrate thatthe performance of our 6 DOF tracking competes with RGB-D tracking algorithms.We compare favorably against strong classic and deep learning powered densedepth algorithms.