The estimation of 3D human motion from video has progressed rapidly but
current methods still have several key limitations. First, most methods
estimate the human in camera coordinates. Second, prior work on estimating
humans in global coordinates often assumes a flat ground plane and produces
foot sliding. Third, the most accurate methods rely on computationally
expensive optimization pipelines, limiting their use to offline applications.
Finally, existing video-based methods are surprisingly less accurate than
single-frame methods. We address these limitations with WHAM (World-grounded
Humans with Accurate Motion), which accurately and efficiently reconstructs 3D
human motion in a global coordinate system from video. WHAM learns to lift 2D
keypoint sequences to 3D using motion capture data and fuses this with video
features, integrating motion context and visual information. WHAM exploits
camera angular velocity estimated from a SLAM method together with human motion
to estimate the body's global trajectory. We combine this with a contact-aware
trajectory refinement method that lets WHAM capture human motion in diverse
conditions, such as climbing stairs. WHAM outperforms all existing 3D human
motion recovery methods across multiple in-the-wild benchmarks. Code will be
available for research purposes at this http URL

通过使用运动捕捉数据将 2D 关键点序列提升到 3D，并将其与视频特征融合，WHAM 能够准确高效地从视频中重建 3D 人体动作，并利用摄像机角速度和人体运动估计了身体的全局轨迹，同时结合了一个联系感知的轨迹细化方法，使 WHAM 能够在不同情况下捕捉人体运动，如爬楼梯等。WHAM 在多个野外基准测试中优于所有现有的 3D 人体运动恢复方法。

使用准确的 3D 动作重建基于世界环境的人类

WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion

Despite significant progress in single image-based 3D human mesh recovery,
accurately and smoothly recovering 3D human motion from a video remains
challenging. Existing video-based methods generally recover human mesh by
estimating the complex pose and shape parameters from coupled image features,
whose high complexity and low representation ability often result in
inconsistent pose motion and limited shape patterns. To alleviate this issue,
we introduce 3D pose as the intermediary and propose a Pose and Mesh
Co-Evolution network (PMCE) that decouples this task into two parts: 1)
video-based 3D human pose estimation and 2) mesh vertices regression from the
estimated 3D pose and temporal image feature. Specifically, we propose a
two-stream encoder that estimates mid-frame 3D pose and extracts a temporal
image feature from the input image sequence. In addition, we design a
co-evolution decoder that performs pose and mesh interactions with the
image-guided Adaptive Layer Normalization (AdaLN) to make pose and mesh fit the
human body shape. Extensive experiments demonstrate that the proposed PMCE
outperforms previous state-of-the-art methods in terms of both per-frame
accuracy and temporal consistency on three benchmark datasets: 3DPW, Human3.6M,
and MPI-INF-3DHP. Our code is available at this https URL

提出了一种 Pose and Mesh Co-Evolution 网络（PMCE），通过将任务分解为视频中基于 3D 人体姿势估计和来自估计的 3D 姿势和时间图像特征的网格顶点回归，解决了从视频中准确、平滑地恢复 3D 人体运动的问题。实验证明，该方法在三个基准数据集（3DPW、Human3.6M 和 MPI-INF-3DHP）上在逐帧准确度和时间一致性方面优于先前的最先进方法。

从视频中估计 3D 人体姿态和网格的共同演化

Co-Evolution of Pose and Mesh for 3D Human Body Estimation from Video

Despite the recent success of single image-based 3D human pose and shape
estimation methods, recovering temporally consistent and smooth 3D human motion
from a video is still challenging. Several video-based methods have been
proposed; however, they fail to resolve the single image-based methods'
temporal inconsistency issue due to a strong dependency on a static feature of
the current frame. In this regard, we present a temporally consistent mesh
recovery system (TCMR). It effectively focuses on the past and future frames'
temporal information without being dominated by the current static feature. Our
TCMR significantly outperforms previous video-based methods in temporal
consistency with better per-frame 3D pose and shape accuracy. We also release
the codes. For the demo video, see this https URL For the codes,
see this https URL

该研究提出了一种基于视频的 3D 人体运动重建系统，该系统有效聚焦于过去和未来帧的时间信息，从而更好地解决了单张图像方法的时间不一致性问题。通过在帧间进行时间一致性的处理，该系统达到了更好的运动姿态和形状精度。