Abstract
While supervised object detection methods achieve impressive accuracy, theygeneralize poorly to images whose appearance significantly differs from thedata they have been trained on. To address this in scenarios where annotatingdata is prohibitively expensive, we introduce a self-supervised approach toobject detection and segmentation, able to work with monocular images capturedwith a moving camera. At the heart of our approach lies the observation thatsegmentation and background reconstruction are linked tasks, and the idea that,because we observe a structured scene, background regions can be re-synthesizedfrom their surroundings, whereas regions depicting the object cannot. Wetherefore encode this intuition as a self-supervised loss function that weexploit to train a proposal-based segmentation network. To account for thediscrete nature of object proposals, we develop a Monte Carlo-based trainingstrategy that allows us to explore the large space of object proposals. Ourexperiments demonstrate that our approach yields accurate detections andsegmentations in images that visually depart from those of standard benchmarks,outperforming existing self-supervised methods and approaching weaklysupervised ones that exploit large annotated datasets.