We present Score-Guided Human Mesh Recovery (ScoreHMR), an approach for
solving inverse problems for 3D human pose and shape reconstruction. These
inverse problems involve fitting a human body model to image observations,
traditionally solved through optimization techniques. ScoreHMR mimics model
fitting approaches, but alignment with the image observation is achieved
through score guidance in the latent space of a diffusion model. The diffusion
model is trained to capture the conditional distribution of the human model
parameters given an input image. By guiding its denoising process with a
task-specific score, ScoreHMR effectively solves inverse problems for various
applications without the need for retraining the task-agnostic diffusion model.
We evaluate our approach on three settings/applications. These are: (i)
single-frame model fitting; (ii) reconstruction from multiple uncalibrated
views; (iii) reconstructing humans in video sequences. ScoreHMR consistently
outperforms all optimization baselines on popular benchmarks across all
settings. We make our code and models available at the
this https URL

我们提出了一种基于得分指导的人体网格恢复（ScoreHMR）方法，用于解决三维人体姿势和形状重建的反问题。

基于评分引导的扩散方法用于三维人体恢复

Score-Guided Diffusion for 3D Human Recovery

The accuracy of 3D Human Pose and Shape reconstruction (HPS) from an image is
progressively improving. Yet, no known method is robust across all image
distortion. To address issues due to variations of camera poses, we introduce
SHARE, a novel fine-tuning method that utilizes adversarial data augmentation
to enhance the robustness of existing HPS techniques. We perform a
comprehensive analysis on the impact of camera poses on HPS reconstruction
outcomes. We first generated large-scale image datasets captured systematically
from diverse camera perspectives. We then established a mapping between camera
poses and reconstruction errors as a continuous function that characterizes the
relationship between camera poses and HPS quality. Leveraging this
representation, we introduce RoME (Regions of Maximal Error), a novel sampling
technique for our adversarial fine-tuning method.
The SHARE framework is generalizable across various single-view HPS methods
and we demonstrate its performance on HMR, SPIN, PARE, CLIFF and ExPose. Our
results illustrate a reduction in mean joint errors across single-view HPS
techniques, for images captured from multiple camera positions without
compromising their baseline performance. In many challenging cases, our method
surpasses the performance of existing models, highlighting its practical
significance for diverse real-world applications.

通过引入 SHARE 方法，利用对抗性数据增强提高现有 HPS 技术的稳健性，我们展示了该方法在多个单视图 HPS 技术上的性能，对于来自多个摄像机位置拍摄的图像降低了关节平均误差，且在许多挑战性案例中超过了现有模型的性能，凸显了其在多样的实际应用中的重要意义。