Whole-body mesh recovery aims to estimate the 3D human body, face, and hands
parameters from a single image. It is challenging to perform this task with a
single network due to resolution issues, i.e., the face and hands are usually
located in extremely small regions. Existing works usually detect hands and
faces, enlarge their resolution to feed in a specific network to predict the
parameter, and finally fuse the results. While this copy-paste pipeline can
capture the fine-grained details of the face and hands, the connections between
different parts cannot be easily recovered in late fusion, leading to
implausible 3D rotation and unnatural pose. In this work, we propose a
one-stage pipeline for expressive whole-body mesh recovery, named OSX, without
separate networks for each part. Specifically, we design a Component Aware
Transformer (CAT) composed of a global body encoder and a local face/hand
decoder. The encoder predicts the body parameters and provides a high-quality
feature map for the decoder, which performs a feature-level upsample-crop
scheme to extract high-resolution part-specific features and adopt
keypoint-guided deformable attention to estimate hand and face precisely. The
whole pipeline is simple yet effective without any manual post-processing and
naturally avoids implausible prediction. Comprehensive experiments demonstrate
the effectiveness of OSX. Lastly, we build a large-scale Upper-Body dataset
(UBody) with high-quality 2D and 3D whole-body annotations. It contains persons
with partially visible bodies in diverse real-life scenarios to bridge the gap
between the basic task and downstream applications.

这篇论文提出了一种名为 OSX 的一阶段人体网格恢复技术，通过使用全局体编码器和本地面部 / 手部解码器组成的组件感知变压器（CAT），预测身体参数并提供高质量的特征图为解码器进行特征级上采样 - 裁剪方案，从而提取高分辨率的部分特定特征，并采用关键点引导的可变形注意力精确估计手部和面部。该技术简单而有效，没有任何手动后处理，并自然地避免了不合理的预测。同时，该论文还构建了一个大规模的上半身数据集（UBody），包含在不同真实场景下出现的在部分可见身体上的人，以弥合基础任务与下游应用之间的差距。