Abstract
Reconstructing the geometry and appearance of objects from photographs takenin different environments is difficult as the illumination and therefore theobject appearance vary across captured images. This is particularly challengingfor more specular objects whose appearance strongly depends on the viewingdirection. Some prior approaches model appearance variation across images usinga per-image embedding vector, while others use physically-based rendering torecover the materials and per-image illumination. Such approaches fail atfaithfully recovering view-dependent appearance given significant variation ininput illumination and tend to produce mostly diffuse results. We present anapproach that reconstructs objects from images taken under differentilluminations by first relighting the images under a single referenceillumination with a multiview relighting diffusion model and thenreconstructing the object's geometry and appearance with a radiance fieldarchitecture that is robust to the small remaining inconsistencies among therelit images. We validate our proposed approach on both synthetic and realdatasets and demonstrate that it greatly outperforms existing techniques atreconstructing high-fidelity appearance from images taken under extremeillumination variation. Moreover, our approach is particularly effective atrecovering view-dependent "shiny" appearance which cannot be reconstructed byprior methods.