Weakly Supervised Learning of Rigid 3D Scene Flow

  • 2021-02-17 18:58:02
  • Zan Gojcic, Or Litany, Andreas Wieser, Leonidas J. Guibas, Tolga Birdal
  • 31


We propose a data-driven scene flow estimation algorithm exploiting theobservation that many 3D scenes can be explained by a collection of agentsmoving as rigid bodies. At the core of our method lies a deep architecture ableto reason at the \textbf{object-level} by considering 3D scene flow inconjunction with other 3D tasks. This object level abstraction, enables us torelax the requirement for dense scene flow supervision with simpler binarybackground segmentation mask and ego-motion annotations. Our mild supervisionrequirements make our method well suited for recently released massive datacollections for autonomous driving, which do not contain dense scene flowannotations. As output, our model provides low-level cues like pointwise flowand higher-level cues such as holistic scene understanding at the level ofrigid objects. We further propose a test-time optimization refining thepredicted rigid scene flow. We showcase the effectiveness and generalizationcapacity of our method on four different autonomous driving datasets. Werelease our source code and pre-trained models under\url{github.com/zgojcic/Rigid3DSceneFlow}.


Quick Read (beta)

loading the full paper ...