Abstract
Motion, measured via optical flow, provides a powerful cue to discover andlearn objects in images and videos. However, compared to using appearance, ithas some blind spots, such as the fact that objects become invisible if they donot move. In this work, we propose an approach that combines the strengths ofmotion-based and appearance-based segmentation. We propose to supervise animage segmentation network, tasking it with predicting regions that are likelyto contain simple motion patterns, and thus likely to correspond to objects. Weapply this network in two modes. In the unsupervised video segmentation mode,the network is trained on a collection of unlabelled videos, using the learningprocess itself as an algorithm to segment these videos. In the unsupervisedimage segmentation model, the network is learned using videos and applied tosegment independent still images. With this, we obtain strong empirical resultsin unsupervised video and image segmentation, significantly outperforming thestate of the art on benchmarks such as DAVIS, sometimes with a $5\%$ IoU gap.