Abstract
Neural Radiance Fields (NeRF) are able to reconstruct scenes withunprecedented fidelity, and various recent works have extended NeRF to handledynamic scenes. A common approach to reconstruct such non-rigid scenes isthrough the use of a learned deformation field mapping from coordinates in eachinput image into a canonical template coordinate space. However, thesedeformation-based approaches struggle to model changes in topology, astopological changes require a discontinuity in the deformation field, but thesedeformation fields are necessarily continuous. We address this limitation bylifting NeRFs into a higher dimensional space, and by representing the 5Dradiance field corresponding to each individual input image as a slice throughthis "hyper-space". Our method is inspired by level set methods, which modelthe evolution of surfaces as slices through a higher dimensional surface. Weevaluate our method on two tasks: (i) interpolating smoothly between "moments",i.e., configurations of the scene, seen in the input images while maintainingvisual plausibility, and (ii) novel-view synthesis at fixed moments. We showthat our method, which we dub HyperNeRF, outperforms existing methods on bothtasks by significant margins. Compared to Nerfies, HyperNeRF reduces averageerror rates by 8.6% for interpolation and 8.8% for novel-view synthesis, asmeasured by LPIPS.