Abstract
Parametric 3D models have enabled a wide variety of tasks in computergraphics and vision, such as modeling human bodies, faces, and hands. However,the construction of these parametric models is often tedious, as it requiresheavy manual tweaking, and they struggle to represent additional complexity anddetails such as wrinkles or clothing. To this end, we propose Neural ParametricModels (NPMs), a novel, learned alternative to traditional, parametric 3Dmodels, which does not require hand-crafted, object-specific constraints. Inparticular, we learn to disentangle 4D dynamics into latent-spacerepresentations of shape and pose, leveraging the flexibility of recentdevelopments in learned implicit functions. Crucially, once learned, our neuralparametric models of shape and pose enable optimization over the learned spacesto fit to new observations, similar to the fitting of a traditional parametricmodel, e.g., SMPL. This enables NPMs to achieve a significantly more accurateand detailed representation of observed deformable sequences. We show that NPMsimprove notably over both parametric and non-parametric state of the art inreconstruction and tracking of monocular depth sequences of clothed humans andhands. Latent-space interpolation as well as shape / pose transfer experimentsfurther demonstrate the usefulness of NPMs.