Abstract
Volumetric neural rendering methods like NeRF generate high-quality viewsynthesis results but are optimized per-scene leading to prohibitivereconstruction time. On the other hand, deep multi-view stereo methods canquickly reconstruct scene geometry via direct network inference. Point-NeRFcombines the advantages of these two approaches by using neural 3D pointclouds, with associated neural features, to model a radiance field. Point-NeRFcan be rendered efficiently by aggregating neural point features near scenesurfaces, in a ray marching-based rendering pipeline. Moreover, Point-NeRF canbe initialized via direct inference of a pre-trained deep network to produce aneural point cloud; this point cloud can be finetuned to surpass the visualquality of NeRF with 30X faster training time. Point-NeRF can be combined withother 3D reconstruction methods and handles the errors and outliers in suchmethods via a novel pruning and growing mechanism.