Abstract
We propose Probabilistic Warp Consistency, a weakly-supervised learningobjective for semantic matching. Our approach directly supervises the densematching scores predicted by the network, encoded as a conditional probabilitydistribution. We first construct an image triplet by applying a known warp toone of the images in a pair depicting different instances of the same objectclass. Our probabilistic learning objectives are then derived using theconstraints arising from the resulting image triplet. We further account forocclusion and background clutter present in real image pairs by extending ourprobabilistic output space with a learnable unmatched state. To supervise it,we design an objective between image pairs depicting different object classes.We validate our method by applying it to four recent semantic matchingarchitectures. Our weakly-supervised approach sets a new state-of-the-art onfour challenging semantic matching benchmarks. Lastly, we demonstrate that ourobjective also brings substantial improvements in the strongly-supervisedregime, when combined with keypoint annotations.