RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs

  • 2021-12-01 18:59:46
  • Michael Niemeyer, Jonathan T. Barron, Ben Mildenhall, Mehdi S. M. Sajjadi, Andreas Geiger, Noha Radwan
  • 40

Abstract

Neural Radiance Fields (NeRF) have emerged as a powerful representation forthe task of novel view synthesis due to their simplicity and state-of-the-artperformance. Though NeRF can produce photorealistic renderings of unseenviewpoints when many input views are available, its performance dropssignificantly when this number is reduced. We observe that the majority ofartifacts in sparse input scenarios are caused by errors in the estimated scenegeometry, and by divergent behavior at the start of training. We address thisby regularizing the geometry and appearance of patches rendered from unobservedviewpoints, and annealing the ray sampling space during training. Weadditionally use a normalizing flow model to regularize the color of unobservedviewpoints. Our model outperforms not only other methods that optimize over asingle scene, but in many cases also conditional models that are extensivelypre-trained on large multi-view datasets.

 

Quick Read (beta)

loading the full paper ...