MELON: NeRF with Unposed Images Using Equivalence Class Estimation

  • 2023-03-14 18:33:39
  • Axel Levy, Mark Matthews, Matan Sela, Gordon Wetzstein, Dmitry Lagun
  • 34


Neural radiance fields enable novel-view synthesis and scene reconstructionwith photorealistic quality from a few images, but require known and accuratecamera poses. Conventional pose estimation algorithms fail on smooth orself-similar scenes, while methods performing inverse rendering from unposedviews require a rough initialization of the camera orientations. The maindifficulty of pose estimation lies in real-life objects being almost invariantunder certain transformations, making the photometric distance between renderedviews non-convex with respect to the camera parameters. Using an equivalencerelation that matches the distribution of local minima in camera space, wereduce this space to its quotient set, in which pose estimation becomes a moreconvex problem. Using a neural-network to regularize pose estimation, wedemonstrate that our method - MELON - can reconstruct a neural radiance fieldfrom unposed images with state-of-the-art accuracy while requiring ten timesfewer views than adversarial approaches.


Quick Read (beta)

loading the full paper ...