For a long time, in the field of reconstructing 3D human bodies from
monocular images, most methods opted to simplify the task by minimizing the
influence of the camera. Using a coarse focal length setting results in the
reconstructed bodies not aligning well with distorted images. Ignoring camera
rotation leads to an unrealistic reconstructed body pose in world space.
Consequently, existing methods' application scenarios are confined to
controlled environments. And they struggle to achieve accurate and reasonable
reconstruction in world space when confronted with complex and diverse
in-the-wild images. To address the above issues, we propose W-HMR, which
decouples global body recovery into camera calibration, local body recovery and
global body orientation correction. We design the first weak-supervised camera
calibration method for body distortion, eliminating dependence on focal length
labels and achieving finer mesh-image alignment. We propose a novel orientation
correction module to allow the reconstructed human body to remain normal in
world space. Decoupling body orientation and body pose enables our model to
consider the accuracy in camera coordinate and the reasonableness in world
coordinate simultaneously, expanding the range of applications. As a result,
W-HMR achieves high-quality reconstruction in dual coordinate systems,
particularly in challenging scenes. Codes will be released on
this https URL after publication.

通过解耦全局身体恢复为相机校准、局部身体恢复和全局身体方向校正，我们的模型能够同时考虑相机坐标的准确性和世界坐标的合理性，在双坐标系中实现高质量的重建，尤其是在具有挑战性的场景中。

W-HMR: 弱监督相机标定和方位校正下的人体网格恢复

W-HMR: Human Mesh Recovery in World Space with Weak-supervised Camera  Calibration and Orientation Correction

We present PyMAF-X, a regression-based approach to recovering a full-body
parametric model from a single image. This task is very challenging since minor
parametric deviation may lead to noticeable misalignment between the estimated
mesh and the input image. Moreover, when integrating part-specific estimations
to the full-body model, existing solutions tend to either degrade the alignment
or produce unnatural wrist poses. To address these issues, we propose a
Pyramidal Mesh Alignment Feedback (PyMAF) loop in our regression network for
well-aligned human mesh recovery and extend it as PyMAF-X for the recovery of
expressive full-body models. The core idea of PyMAF is to leverage a feature
pyramid and rectify the predicted parameters explicitly based on the mesh-image
alignment status. Specifically, given the currently predicted parameters,
mesh-aligned evidence will be extracted from finer-resolution features
accordingly and fed back for parameter rectification. To enhance the alignment
perception, an auxiliary dense supervision is employed to provide mesh-image
correspondence guidance while spatial alignment attention is introduced to
enable the awareness of the global contexts for our network. When extending
PyMAF for full-body mesh recovery, an adaptive integration strategy is proposed
in PyMAF-X to produce natural wrist poses while maintaining the well-aligned
performance of the part-specific estimations. The efficacy of our approach is
validated on several benchmark datasets for body-only and full-body mesh
recovery, where PyMAF and PyMAF-X effectively improve the mesh-image alignment
and achieve new state-of-the-art results. The project page with code and video
results can be found at this https URL

本论文提出了一种基于回归的方法 PyMAF-X，通过 Pyramidal Mesh Alignment Feedback 循环实现了全身参数模型的从单张图片中恢复，并提出自适应集成策略以产生自然的手腕姿势，保持了部分特定估计的对齐性能。在多个基准数据集上验证了我们方法的有效性，并取得了表现优异的结果。

PyMAF-X: 从单目图像实现全身模型回归的良好对齐

PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular  Images

3D hand-mesh reconstruction from RGB images facilitates many applications,
including augmented reality (AR). However, this requires not only real-time
speed and accurate hand pose and shape but also plausible mesh-image alignment.
While existing works already achieve promising results, meeting all three
requirements is very challenging. This paper presents a novel pipeline by
decoupling the hand-mesh reconstruction task into three stages: a joint stage
to predict hand joints and segmentation; a mesh stage to predict a rough hand
mesh; and a refine stage to fine-tune it with an offset mesh for mesh-image
alignment. With careful design in the network structure and in the loss
functions, we can promote high-quality finger-level mesh-image alignment and
drive the models together to deliver real-time predictions. Extensive
quantitative and qualitative results on benchmark datasets demonstrate that the
quality of our results outperforms the state-of-the-art methods on
hand-mesh/pose precision and hand-image alignment. In the end, we also showcase
several real-time AR scenarios.

该论文提出了一种基于三个阶段的新型流程，通过联合阶段、网格阶段和细化阶段，实现准确的手部姿势和形状的 3D 网格重建，从而达到较高的手 - 图像对齐精度和实时性。该流程在基准数据集上具有优越的结果，证明了其高质量的手 - 网格 / 姿态预测和图像对齐性能。