Abstract
We present the first system for physically based, neural inverse renderingfrom multi-viewpoint videos of propagating light. Our approach relies on atime-resolved extension of neural radiance caching -- a technique thataccelerates inverse rendering by storing infinite-bounce radiance arriving atany point from any direction. The resulting model accurately accounts fordirect and indirect light transport effects and, when applied to capturedmeasurements from a flash lidar system, enables state-of-the-art 3Dreconstruction in the presence of strong indirect light. Further, wedemonstrate view synthesis of propagating light, automatic decomposition ofcaptured measurements into direct and indirect components, as well as novelcapabilities such as multi-view time-resolved relighting of captured scenes.