We present a method to capture temporally coherent dynamic clothing
deformation from a monocular RGB video input. In contrast to the existing
literature, our method does not require a pre-scanned personalized mesh
template, and thus can be applied to in-the-wild videos. To constrain the
output to a valid deformation space, we build statistical deformation models
for three types of clothing: T-shirt, short pants and long pants. A
differentiable renderer is utilized to align our captured shapes to the input
frames by minimizing the difference in both silhouette, segmentation, and
texture. We develop a UV texture growing method which expands the visible
texture region of the clothing sequentially in order to minimize drift in
deformation tracking. We also extract fine-grained wrinkle detail from the
input videos by fitting the clothed surface to the normal maps estimated by a
convolutional neural network. Our method produces temporally coherent
reconstruction of body and clothing from monocular video. We demonstrate
successful clothing capture results from a variety of challenging videos.
Extensive quantitative experiments demonstrate the effectiveness of our method
on metrics including body pose error and surface reconstruction error of the
clothing.

本文提出了一种从单目 RGB 视频输入中捕捉时间连贯的动态服装变形的方法，该方法不需要预先扫描个性化网格模板，可以用于野外视频，并建立三种类型的服装的统计变形模型，利用可微分渲染器将捕获到的形状与输入帧对齐，并提取出细节纹理等信息。经过大量实验表明，该方法可以成功地从各种具有挑战性的视频中捕获服装，同时也能够在衣服表面重建误差等指标上达到有效的效果。