Abstract
Keypoints are what enable Structure-from-Motion (SfM) systems to scale tothousands of images. However, designing a keypoint detection objective is anon-trivial task, as SfM is non-differentiable. Typically, an auxiliaryobjective involving a descriptor is optimized. This however induces adependency on the descriptor, which is undesirable. In this paper we propose afully self-supervised and descriptor-free objective for keypoint detection,through reinforcement learning. To ensure training does not degenerate, weleverage a balanced top-K sampling strategy. While this already producescompetitive models, we find that two qualitatively different types of detectorsemerge, which are only able to detect light and dark keypoints respectively. Toremedy this, we train a third detector, DaD, that optimizes theKullback-Leibler divergence of the pointwise maximum of both light and darkdetectors. Our approach significantly improve upon SotA across a range ofbenchmarks. Code and model weights are publicly available athttps://github.com/parskatt/dad