The recovery of 3D human mesh from monocular images has significantly been
developed in recent years. However, existing models usually ignore spatial and
temporal information, which might lead to mesh and image misalignment and
temporal discontinuity. For this reason, we propose a novel Spatio-Temporal
Alignment Fusion (STAF) model. As a video-based model, it leverages coherence
clues from human motion by an attention-based Temporal Coherence Fusion Module
(TCFM). As for spatial mesh-alignment evidence, we extract fine-grained local
information through predicted mesh projection on the feature maps. Based on the
spatial features, we further introduce a multi-stage adjacent Spatial Alignment
Fusion Module (SAFM) to enhance the feature representation of the target frame.
In addition to the above, we propose an Average Pooling Module (APM) to allow
the model to focus on the entire input sequence rather than just the target
frame. This method can remarkably improve the smoothness of recovery results
from video. Extensive experiments on 3DPW, MPII3D, and H36M demonstrate the
superiority of STAF. We achieve a state-of-the-art trade-off between precision
and smoothness. Our code and more video results are on the project page
this https URL

提出了一种基于视频的 Spatio-Temporal Alignment Fusion (STAF) 模型，通过关注基于注意力的 Temporal Coherence Fusion Module (TCFM) 中的人体运动的相干线索，以及通过在特征图上预测网格投影来提取细粒度局部信息的 Spatial Alignment Fusion Module (SAFM)，进一步引入多阶段邻近空间对齐融合模块以增强目标帧的特征表示，同时使用 Average Pooling Module (APM) 提高对整个输入序列的关注，从而显著提高了从视频中恢复结果的平滑度。通过在 3DPW、MPII3D 和 H36M 上进行了大量实验证明了 STAF 的优越性，实现了在精度和平滑度之间的最新权衡。

STAF：视频中基于时空对齐融合的三维人体网格恢复

STAF: 3D Human Mesh Recovery from Video with Spatio-Temporal Alignment  Fusion

Recovering a 3D human mesh from a single RGB image is a challenging task due
to depth ambiguity and self-occlusion, resulting in a high degree of
uncertainty. Meanwhile, diffusion models have recently seen much success in
generating high-quality outputs by progressively denoising noisy inputs.
Inspired by their capability, we explore a diffusion-based approach for human
mesh recovery, and propose a Human Mesh Diffusion (HMDiff) framework which
frames mesh recovery as a reverse diffusion process. We also propose a
Distribution Alignment Technique (DAT) that injects input-specific distribution
information into the diffusion process, and provides useful prior knowledge to
simplify the mesh recovery task. Our method achieves state-of-the-art
performance on three widely used datasets. Project page:
this https URL

通过扩散过程恢复人体网格，以解决深度模糊和自遮挡问题，并提出注入特定输入分布信息的分布对齐技术（DAT），通过简化网格恢复任务提供有用的先验知识。

分布对齐扩散的人体网格恢复

Distribution-Aligned Diffusion for Human Mesh Recovery

Recovering 3D human mesh in the wild is greatly challenging as in-the-wild
(ITW) datasets provide only 2D pose ground truths (GTs). Recently, 3D
pseudo-GTs have been widely used to train 3D human mesh estimation networks as
the 3D pseudo-GTs enable 3D mesh supervision when training the networks on ITW
datasets. However, despite the great potential of the 3D pseudo-GTs, there has
been no extensive analysis that investigates which factors are important to
make more beneficial 3D pseudo-GTs. In this paper, we provide three recipes to
obtain highly beneficial 3D pseudo-GTs of ITW datasets. The main challenge is
that only 2D-based weak supervision is allowed when obtaining the 3D
pseudo-GTs. Each of our three recipes addresses the challenge in each aspect:
depth ambiguity, sub-optimality of weak supervision, and implausible
articulation. Experimental results show that simply re-training
state-of-the-art networks with our new 3D pseudo-GTs elevates their performance
to the next level without bells and whistles. The 3D pseudo-GT is publicly
available in this https URL

本文研究在野生环境下恢复 3D 人体网格的挑战，提供三种方法得到高效 3D 假的 GT。

更好的 3D 假标注配方：野外 3D 人体网格估计的三种方法

Three Recipes for Better 3D Pseudo-GTs of 3D Human Mesh Estimation in  the Wild

We consider the problem of recovering a single person's 3D human mesh from
in-the-wild crowded scenes. While much progress has been in 3D human mesh
estimation, existing methods struggle when test input has crowded scenes. The
first reason for the failure is a domain gap between training and testing data.
A motion capture dataset, which provides accurate 3D labels for training, lacks
crowd data and impedes a network from learning crowded scene-robust image
features of a target person. The second reason is a feature processing that
spatially averages the feature map of a localized bounding box containing
multiple people. Averaging the whole feature map makes a target person's
feature indistinguishable from others. We present 3DCrowdNet that firstly
explicitly targets in-the-wild crowded scenes and estimates a robust 3D human
mesh by addressing the above issues. First, we leverage 2D human pose
estimation that does not require a motion capture dataset with 3D labels for
training and does not suffer from the domain gap. Second, we propose a
joint-based regressor that distinguishes a target person's feature from others.
Our joint-based regressor preserves the spatial activation of a target by
sampling features from the target's joint locations and regresses human model
parameters. As a result, 3DCrowdNet learns target-focused features and
effectively excludes the irrelevant features of nearby persons. We conduct
experiments on various benchmarks and prove the robustness of 3DCrowdNet to the
in-the-wild crowded scenes both quantitatively and qualitatively. The code is
available at this https URL

该研究论文提出了一种名为 3DCrowdNet 的深度学习模型，利用 2D 人体姿态估计和基于关节回归的方法，从拥挤的实景图像中恢复出单个人的三维人体模型。

学习从复杂野外场景中恢复鲁棒的三维人体网格

Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded  Scenes

We consider the problem of estimating a parametric model of 3D human mesh
from a single image. While there has been substantial recent progress in this
area with direct regression of model parameters, these methods only implicitly
exploit the human body kinematic structure, leading to sub-optimal use of the
model prior. In this work, we address this gap by proposing a new technique for
regression of human parametric model that is explicitly informed by the known
hierarchical structure, including joint interdependencies of the model. This
results in a strong prior-informed design of the regressor architecture and an
associated hierarchical optimization that is flexible to be used in conjunction
with the current standard frameworks for 3D human mesh recovery. We demonstrate
these aspects by means of extensive experiments on standard benchmark datasets,
showing how our proposed new design outperforms several existing and popular
methods, establishing new state-of-the-art results. By considering joint
interdependencies, our method is equipped to infer joints even under data
corruptions, which we demonstrate by conducting experiments under varying
degrees of occlusion.

提出了一种基于已知层次结构的人体参数化模型回归新技术，旨在更加灵活应用于当前标准的三维人体网格恢复框架，并通过实验数据证明其性能优于现有方法，能在数据损坏下推断关节。