NeRFMeshing: Distilling Neural Radiance Fields into Geometrically-Accurate 3D Meshes

  • 2023-03-16 17:06:03
  • Marie-Julie Rakotosaona, Fabian Manhardt, Diego Martin Arroyo, Michael Niemeyer, Abhijit Kundu, Federico Tombari
  • 113

Abstract

With the introduction of Neural Radiance Fields (NeRFs), novel view synthesishas recently made a big leap forward. At the core, NeRF proposes that each 3Dpoint can emit radiance, allowing to conduct view synthesis usingdifferentiable volumetric rendering. While neural radiance fields canaccurately represent 3D scenes for computing the image rendering, 3D meshes arestill the main scene representation supported by most computer graphics andsimulation pipelines, enabling tasks such as real time rendering andphysics-based simulations. Obtaining 3D meshes from neural radiance fieldsstill remains an open challenge since NeRFs are optimized for view synthesis,not enforcing an accurate underlying geometry on the radiance field. We thuspropose a novel compact and flexible architecture that enables easy 3D surfacereconstruction from any NeRF-driven approach. Upon having trained the radiancefield, we distill the volumetric 3D representation into a Signed SurfaceApproximation Network, allowing easy extraction of the 3D mesh and appearance.Our final 3D mesh is physically accurate and can be rendered in real time on anarray of devices.

 

Quick Read (beta)

loading the full paper ...