Self-supervised Learning for Single View Depth and Surface Normal Estimation

  • 2019-03-01 00:07:12
  • Huangying Zhan, Chamara Saroj Weerasekera, Ravi Garg, Ian Reid
  • 22


In this work we present a self-supervised learning framework tosimultaneously train two Convolutional Neural Networks (CNNs) to predict depthand surface normals from a single image. In contrast to most existingframeworks which represent outdoor scenes as fronto-parallel planes atpiece-wise smooth depth, we propose to predict depth with surface orientationwhile assuming that natural scenes have piece-wise smooth normals. We show thata simple depth-normal consistency as a soft-constraint on the predictions issufficient and effective for training both these networks simultaneously. Thetrained normal network provides state-of-the-art predictions while the depthnetwork, relying on much realistic smooth normal assumption, outperforms thetraditional self-supervised depth prediction network by a large margin on theKITTI benchmark. Demo video:


Introduction (beta)



Conclusion (beta)