HexPlane: A Fast Representation for Dynamic Scenes

  • 2023-01-23 18:59:25
  • Ang Cao, Justin Johnson
  • 55

Abstract

Modeling and re-rendering dynamic 3D scenes is a challenging task in 3Dvision. Prior approaches build on NeRF and rely on implicit representations.This is slow since it requires many MLP evaluations, constraining real-worldapplications. We show that dynamic 3D scenes can be explicitly represented bysix planes of learned features, leading to an elegant solution we callHexPlane. A HexPlane computes features for points in spacetime by fusingvectors extracted from each plane, which is highly efficient. Pairing aHexPlane with a tiny MLP to regress output colors and training via volumerendering gives impressive results for novel view synthesis on dynamic scenes,matching the image quality of prior work but reducing training time by morethan $100\times$. Extensive ablations confirm our HexPlane design and show thatit is robust to different feature fusion mechanisms, coordinate systems, anddecoding mechanisms. HexPlanes are a simple and effective solution forrepresenting 4D volumes, and we hope they can broadly contribute to modelingspacetime for dynamic 3D scenes.

 

Quick Read (beta)

loading the full paper ...