Learning to regress 3D human body shape and pose (e.g.~SMPL parameters) from monocular images typically exploits losses on 2D keypoints, silhouettes, and/or part-segmentation when 3D training data is not available. Such losses, however, are limited because 2D keypoints do not supervise body shape and segmentations of people in clothing do not match projected minimally-clothed SMPL shapes. To exploit richer image information about clothed people, we introduce higher-level semantic information about clothing to penalize clothed and non-clothed regions of the image differently. To do so, we train a body regressor using a novel Differentiable Semantic Rendering - DSR loss. For Minimally-Clothed regions, we define the DSR-MC loss, which encourages a tight match between a rendered SMPL body and the minimally-clothed regions of the image. For clothed regions, we define the DSR-C loss to encourage the rendered SMPL body to be inside the clothing mask. To ensure end-to-end differentiable training, we learn a semantic clothing prior for SMPL vertices from thousands of clothed human scans. We perform extensive qualitative and quantitative experiments to evaluate the role of clothing semantics on the accuracy of 3D human pose and shape estimation. We outperform all previous state-of-the-art methods on 3DPW and Human3.6M and obtain on par results on MPI-INF-3DHP. Code and trained models are available for research at https://dsr.is.tue.mpg.de/.

利用Differentiable Semantic Rendering（DSR）loss来训练一个人体回归器，从而从单目图像中回归出3D人体形状和姿势。在DSR loss中，利用了服装的语义信息来惩罚有衣和没衣的区域，从而更好的匹配服装区和未穿衣区，同时为了确保可微的训练，从成千上万个衣服扫描中学习SMPL顶点的语义衣服先验。在3DPW和Human3.6M上优于先前的最先进的方法，并在MPI-INF-3DHP上获得了与之相当的结果。

使用可导语义渲染从图像中学习回归人体