NeROIC: Neural Rendering of Objects from Online Image Collections

  • 2022-01-07 16:45:15
  • Zhengfei Kuang, Kyle Olszewski, Menglei Chai, Zeng Huang, Panos Achlioptas, Sergey Tulyakov
  • 91

Abstract

We present a novel method to acquire object representations from online imagecollections, capturing high-quality geometry and material properties ofarbitrary objects from photographs with varying cameras, illumination, andbackgrounds. This enables various object-centric rendering applications such asnovel-view synthesis, relighting, and harmonized background composition fromchallenging in-the-wild input. Using a multi-stage approach extending neuralradiance fields, we first infer the surface geometry and refine the coarselyestimated initial camera parameters, while leveraging coarse foreground objectmasks to improve the training efficiency and geometry quality. We alsointroduce a robust normal estimation technique which eliminates the effect ofgeometric noise while retaining crucial details. Lastly, we extract surfacematerial properties and ambient illumination, represented in sphericalharmonics with extensions that handle transient elements, e.g. sharp shadows.The union of these components results in a highly modular and efficient objectacquisition framework. Extensive evaluations and comparisons demonstrate theadvantages of our approach in capturing high-quality geometry and appearanceproperties useful for rendering applications.

 

Quick Read (beta)

loading the full paper ...