Abstract
Decomposing an object's appearance into representations of its materials andthe surrounding illumination is difficult, even when the object's 3D shape isknown beforehand. This problem is ill-conditioned because diffuse materialsseverely blur incoming light, and is ill-posed because diffuse materials underhigh-frequency lighting can be indistinguishable from shiny materials underlow-frequency lighting. We show that it is possible to recover precisematerials and illumination -- even from diffuse objects -- by exploitingunintended shadows, like the ones cast onto an object by the photographer whomoves around it. These shadows are a nuisance in most previous inverserendering pipelines, but here we exploit them as signals that improveconditioning and help resolve material-lighting ambiguities. We present amethod based on differentiable Monte Carlo ray tracing that uses images of anobject to jointly recover its spatially-varying materials, the surroundingillumination environment, and the shapes of the unseen light occluders whoinadvertently cast shadows upon it.