Abstract
We present a new framework for learning 3D object shapes and densecross-object 3D correspondences from just an unaligned category-specific imagecollection. The 3D shapes are generated implicitly as deformations to acategory-specific signed distance field and are learned in an unsupervisedmanner solely from unaligned image collections without any 3D supervision.Generally, image collections on the internet contain several intra-categorygeometric and topological variations, for example, different chairs can havedifferent topologies, which makes the task of joint shape and correspondenceestimation much more challenging. Because of this, prior works either focus onlearning each 3D object shape individually without modeling cross-instancecorrespondences or perform joint shape and correspondence estimation oncategories with minimal intra-category topological variations. We overcomethese restrictions by learning a topologically-aware implicit deformation fieldthat maps a 3D point in the object space to a higher dimensional point in thecategory-specific canonical space. At inference time, given a single image, wereconstruct the underlying 3D shape by first implicitly deforming each 3D pointin the object space to the learned category-specific canonical space using thetopologically-aware deformation field and then reconstructing the 3D shape as acanonical signed distance field. Both canonical shape and deformation field arelearned end-to-end in an inverse-graphics fashion using a learned recurrent raymarcher (SRN) as a differentiable rendering module. Our approach, dubbed TARS,achieves state-of-the-art reconstruction fidelity on several datasets:ShapeNet, Pascal3D+, CUB, and Pix3D chairs. Result videos and code athttps://shivamduggal4.github.io/tars-3D/