Abstract
Establishing robust and accurate correspondences between a pair of images isa long-standing computer vision problem with numerous applications. Whileclassically dominated by sparse methods, emerging dense approaches offer acompelling alternative paradigm that avoids the keypoint detection step.However, dense flow estimation is often inaccurate in the case of largedisplacements, occlusions, or homogeneous regions. In order to apply densemethods to real-world applications, such as pose estimation, imagemanipulation, or 3D reconstruction, it is therefore crucial to estimate theconfidence of the predicted matches. We propose the Enhanced Probabilistic Dense Correspondence Network, PDC-Net+,capable of estimating accurate dense correspondences along with a reliableconfidence map. We develop a flexible probabilistic approach that jointlylearns the flow prediction and its uncertainty. In particular, we parametrizethe predictive distribution as a constrained mixture model, ensuring bettermodelling of both accurate flow predictions and outliers. Moreover, we developan architecture and an enhanced training strategy tailored for robust andgeneralizable uncertainty prediction in the context of self-supervisedtraining. Our approach obtains state-of-the-art results on multiple challenginggeometric matching and optical flow datasets. We further validate theusefulness of our probabilistic confidence estimation for the tasks of poseestimation, 3D reconstruction, image-based localization, and image retrieval.Code and models are available at https://github.com/PruneTruong/DenseMatching.