3D human pose estimation in outdoor environments has garnered increasing
attention recently. However, prevalent 3D human pose datasets pertaining to
outdoor scenes lack diversity, as they predominantly utilize only one type of
modality (RGB image or pointcloud), and often feature only one individual
within each scene. This limited scope of dataset infrastructure considerably
hinders the variability of available data. In this article, we propose
Human-M3, an outdoor multi-modal multi-view multi-person human pose database
which includes not only multi-view RGB videos of outdoor scenes but also
corresponding pointclouds. In order to obtain accurate human poses, we propose
an algorithm based on multi-modal data input to generate ground truth
annotation. This benefits from robust pointcloud detection and tracking, which
solves the problem of inaccurate human localization and matching ambiguity that
may exist in previous multi-view RGB videos in outdoor multi-person scenes, and
generates reliable ground truth annotations. Evaluation of multiple different
modalities algorithms has shown that this database is challenging and suitable
for future research. Furthermore, we propose a 3D human pose estimation
algorithm based on multi-modal data input, which demonstrates the advantages of
multi-modal data input for 3D human pose estimation. Code and data will be
released on this https URL

提出 Human-M3，一个室外多模态多视角多人姿势数据库，包括多视角 RGB 视频和对应的点云；基于多模态数据输入，提出一种姿势准确的算法以生成地面真实标注，在室外多人场景中解决多视角 RGB 视频中存在的人体定位和匹配模糊问题，并生成可靠的地面真实标注；多种不同模态算法的评估表明该数据库具有挑战性且适用于未来研究；此外，还提出了一种基于多模态数据输入的 3D 人体姿势估计算法，展示了多模态数据输入在 3D 人体姿势估计中的优势。

Human-M3：一个用于户外场景中的多视角多模态三维人体姿态估计的数据集

Human-M3: A Multi-view Multi-modal Dataset for 3D Human Pose Estimation  in Outdoor Scenes

Combining human body models with differentiable rendering has recently
enabled animatable avatars of clothed humans from sparse sets of multi-view RGB
videos. While state-of-the-art approaches achieve realistic appearance with
neural radiance fields (NeRF), the inferred geometry often lacks detail due to
missing geometric constraints. Further, animating avatars in
out-of-distribution poses is not yet possible because the mapping from
observation space to canonical space does not generalize faithfully to unseen
poses. In this work, we address these shortcomings and propose a model to
create animatable clothed human avatars with detailed geometry that generalize
well to out-of-distribution poses. To achieve detailed geometry, we combine an
articulated implicit surface representation with volume rendering. For
generalization, we propose a novel joint root-finding algorithm for
simultaneous ray-surface intersection search and correspondence search. Our
algorithm enables efficient point sampling and accurate point canonicalization
while generalizing well to unseen poses. We demonstrate that our proposed
pipeline can generate clothed avatars with high-quality pose-dependent geometry
and appearance from a sparse set of multi-view RGB videos. Our method achieves
state-of-the-art performance on geometry and appearance reconstruction while
creating animatable avatars that generalize well to out-of-distribution poses
beyond the small number of training poses.

本文提出了一种方法，将人体模型与可微分渲染相结合，从多视点 RGB 视频中创建可动态的穿着衣服的人形化身，其具有细节化的几何结构，并且能够适应超出分布范围的姿势。

ARAH：人体关节 SDF 的可动态渲染

ARAH: Animatable Volume Rendering of Articulated Human SDFs

We propose a new method for reconstructing controllable implicit 3D human
models from sparse multi-view RGB videos. Our method defines the neural scene
representation on the mesh surface points and signed distances from the surface
of a human body mesh. We identify an indistinguishability issue that arises
when a point in 3D space is mapped to its nearest surface point on a mesh for
learning surface-aligned neural scene representation. To address this issue, we
propose projecting a point onto a mesh surface using a barycentric
interpolation with modified vertex normals. Experiments with the ZJU-MoCap and
Human3.6M datasets show that our approach achieves a higher quality in a
novel-view and novel-pose synthesis than existing methods. We also demonstrate
that our method easily supports the control of body shape and clothes. Project
page: this https URL

本研究提出了一种从稀疏的多视角 RGB 视频中重建可控的隐式 3D 人体模型的新方法，使用网格表面点和身体网格的符号距离来定义神经场景表示，并通过改进的顶点法线的重心插值将点投影到网格表面上，实现了更高质量的新视角和新姿势合成，同时支持容易控制的身体形状和服装。