Abstract
Learning methods for relative camera pose estimation have been developedlargely in isolation from classical geometric approaches. The question of howto integrate predictions from deep neural networks (DNNs) and solutions fromgeometric solvers, such as the 5-point algorithm, has as yet remainedunder-explored. In this paper, we present a novel framework that involvesprobabilistic fusion between the two families of predictions during networktraining, with a view to leveraging their complementary benefits in a learnableway. The fusion is achieved by learning the DNN uncertainty under explicitguidance by the geometric uncertainty, thereby learning to take into accountthe geometric solution in relation to the DNN prediction. Our network featuresa self-attention graph neural network, which drives the learning by enforcingstrong interactions between different correspondences and potentially modelingcomplex relationships between points. We propose motion parmeterizationssuitable for learning and show that our method achieves state-of-the-artperformance on the challenging DeMoN and ScanNet datasets. While we focus onrelative pose, we envision that our pipeline is broadly applicable for fusingclassical geometry and deep learning.