Abstract
We propose a differentiable sphere tracing algorithm to bridge the gapbetween inverse graphics methods and the recently proposed deep learning basedimplicit signed distance function. Due to the nature of the implicit function,the rendering process requires tremendous function queries, which isparticularly problematic when the function is represented as a neural network.We optimize both the forward and backward pass of our rendering layer to makeit run efficiently with affordable memory consumption on a commodity graphicscard. Our rendering method is fully differentiable such that losses can bedirectly computed on the rendered 2D observations, and the gradients can bepropagated backward to optimize the 3D geometry. We show that our renderingmethod can effectively reconstruct accurate 3D shapes from various inputs, suchas sparse depth and multi-view images, through inverse optimization. With thegeometry based reasoning, our 3D shape prediction methods show excellentgeneralization capability and robustness against various noise.