Abstract
We aim to recover the geometry of 3D parametric scenes using very few depthmeasurements from low-cost, commercially available time-of-flight sensors.These sensors offer very low spatial resolution (i.e., a single pixel), butimage a wide field-of-view per pixel and capture detailed time-of-flight datain the form of time-resolved photon counts. This time-of-flight data encodesrich scene information and thus enables recovery of simple scenes from sparsemeasurements. We investigate the feasibility of using a distributed set of fewmeasurements (e.g., as few as 15 pixels) to recover the geometry of simpleparametric scenes with a strong prior, such as estimating the 6D pose of aknown object. To achieve this, we design a method that utilizes bothfeed-forward prediction to infer scene parameters, and differentiable renderingwithin an analysis-by-synthesis framework to refine the scene parameterestimate. We develop hardware prototypes and demonstrate that our methodeffectively recovers object pose given an untextured 3D model in bothsimulations and controlled real-world captures, and show promising initialresults for other parametric scenes. We additionally conduct experiments toexplore the limits and capabilities of our imaging solution.