Abstract
We present SCANimate, an end-to-end trainable framework that takes raw 3Dscans of a clothed human and turns them into an animatable avatar. Theseavatars are driven by pose parameters and have realistic clothing that movesand deforms naturally. SCANimate does not rely on a customized mesh template orsurface mesh registration. We observe that fitting a parametric 3D body model,like SMPL, to a clothed human scan is tractable while surface registration ofthe body topology to the scan is often not, because clothing can deviatesignificantly from the body shape. We also observe that articulatedtransformations are invertible, resulting in geometric cycle consistency in theposed and unposed shapes. These observations lead us to a weakly supervisedlearning method that aligns scans into a canonical pose by disentanglingarticulated deformations without template-based surface registration.Furthermore, to complete missing regions in the aligned scans while modelingpose-dependent deformations, we introduce a locally pose-aware implicitfunction that learns to complete and model geometry with learned posecorrectives. In contrast to commonly used global pose embeddings, our localpose conditioning significantly reduces long-range spurious correlations andimproves generalization to unseen poses, especially when training data islimited. Our method can be applied to pose-aware appearance modeling togenerate a fully textured avatar. We demonstrate our approach on variousclothing types with different amounts of training data, outperforming existingsolutions and other variants in terms of fidelity and generality in everysetting. The code is available at https://scanimate.is.tue.mpg.de.