Abstract
We address the unsupervised learning of several interconnected problems inlow-level vision: single view depth prediction, camera motion estimation,optical flow and segmentation of a video into the static scene and movingregions. Our key insight is that these four fundamental vision problems arecoupled and, consequently, learning to solve them together simplifies theproblem because the solutions can reinforce each other by exploiting knowngeometric constraints. In order to model geometric constraints, we introduceAdversarial Collaboration, a framework that facilitates competition andcollaboration between neural networks. We go beyond previous work by exploitinggeometry more explicitly and segmenting the scene into static and movingregions. Adversarial Collaboration works much like expectation-maximization butwith neural networks that act as adversaries, competing to explain pixels thatcorrespond to static or moving regions, and as collaborators through amoderator that assigns pixels to be either static or independently moving. Ournovel method integrates all these problems in a common framework andsimultaneously reasons about the segmentation of the scene into moving objectsand the static background, the camera motion, depth of the static scenestructure, and the optical flow of moving objects. Our model is trained withoutany supervision and achieves state of the art results amongst unsupervisedmethods.