Abstract
The key challenge in learning dense correspondences lies in the lack ofground-truth matches for real image pairs. While photometric consistency lossesprovide unsupervised alternatives, they struggle with large appearance changes,which are ubiquitous in geometric and semantic matching tasks. Moreover,methods relying on synthetic training pairs often suffer from poorgeneralisation to real data. We propose Warp Consistency, an unsupervised learning objective for densecorrespondence regression. Our objective is effective even in settings withlarge appearance and view-point changes. Given a pair of real images, we firstconstruct an image triplet by applying a randomly sampled warp to one of theoriginal images. We derive and analyze all flow-consistency constraints arisingbetween the triplet. From our observations and empirical results, we design ageneral unsupervised objective employing two of the derived constraints. Wevalidate our warp consistency loss by training three recent densecorrespondence networks for the geometric and semantic matching tasks. Ourapproach sets a new state-of-the-art on several challenging benchmarks,including MegaDepth, RobotCar and TSS. Code and models will be released athttps://github.com/PruneTruong/DenseMatching.