Neural Fields in Visual Computing and Beyond

  • 2021-11-22 18:57:51
  • Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, Srinath Sridhar
  • 58


Recent advances in machine learning have created increasing interest insolving visual computing problems using a class of coordinate-based neuralnetworks that parametrize physical properties of scenes or objects across spaceand time. These methods, which we call neural fields, have seen successfulapplication in the synthesis of 3D shapes and image, animation of human bodies,3D reconstruction, and pose estimation. However, due to rapid progress in ashort time, many papers exist but a comprehensive review and formulation of theproblem has not yet emerged. In this report, we address this limitation byproviding context, mathematical grounding, and an extensive review ofliterature on neural fields. This report covers research along two dimensions.In Part I, we focus on techniques in neural fields by identifying commoncomponents of neural field methods, including different representations,architectures, forward mapping, and generalization methods. In Part II, wefocus on applications of neural fields to different problems in visualcomputing, and beyond (e.g., robotics, audio). Our review shows the breadth oftopics already covered in visual computing, both historically and in currentincarnations, demonstrating the improved quality, flexibility, and capabilitybrought by neural fields methods. Finally, we present a companion website thatcontributes a living version of this review that can be continually updated bythe community.


Quick Read (beta)

loading the full paper ...