Learning to regress 3D human body shape and pose (e.g.~SMPL parameters) from
monocular images typically exploits losses on 2D keypoints, silhouettes, and/or
part-segmentation when 3D training data is not available. Such losses, however,
are limited because 2D keypoints do not supervise body shape and segmentations
of people in clothing do not match projected minimally-clothed SMPL shapes. To
exploit richer image information about clothed people, we introduce
higher-level semantic information about clothing to penalize clothed and
non-clothed regions of the image differently. To do so, we train a body
regressor using a novel Differentiable Semantic Rendering - DSR loss. For
Minimally-Clothed regions, we define the DSR-MC loss, which encourages a tight
match between a rendered SMPL body and the minimally-clothed regions of the
image. For clothed regions, we define the DSR-C loss to encourage the rendered
SMPL body to be inside the clothing mask. To ensure end-to-end differentiable
training, we learn a semantic clothing prior for SMPL vertices from thousands
of clothed human scans. We perform extensive qualitative and quantitative
experiments to evaluate the role of clothing semantics on the accuracy of 3D
human pose and shape estimation. We outperform all previous state-of-the-art
methods on 3DPW and Human3.6M and obtain on par results on MPI-INF-3DHP. Code
and trained models are available for research at this https URL

利用 Differentiable Semantic Rendering（DSR）loss 来训练一个人体回归器，从而从单目图像中回归出 3D 人体形状和姿势。在 DSR loss 中，利用了服装的语义信息来惩罚有衣和没衣的区域，从而更好的匹配服装区和未穿衣区，同时为了确保可微的训练，从成千上万个衣服扫描中学习 SMPL 顶点的语义衣服先验。在 3DPW 和 Human3.6M 上优于先前的最先进的方法，并在 MPI-INF-3DHP 上获得了与之相当的结果。