Abstract
We present a method to capture temporally coherent dynamic clothingdeformation from a monocular RGB video input. In contrast to the existingliterature, our method does not require a pre-scanned personalized meshtemplate, and thus can be applied to in-the-wild videos. To constrain theoutput to a valid deformation space, we build statistical deformation modelsfor three types of clothing: T-shirt, short pants and long pants. Adifferentiable renderer is utilized to align our captured shapes to the inputframes by minimizing the difference in both silhouette and texture. We developa UV texture growing method which expands the visible texture region of theclothing sequentially in order to minimize drift in deformation tracking. Wealso extract fine-grained wrinkle detail from the input videos by fitting theclothed surface to the normal maps estimated by a convolutional neural network.Our method produces temporally coherent reconstruction of body and clothingfrom monocular video. We demonstrate successful clothing capture results from avariety of challenging videos. Extensive quantitative experiments demonstratethe effectiveness of our method on metrics including body pose error andsurface reconstruction error of the clothing.