Neural rendering techniques have significantly advanced 3D human body
modeling. However, previous approaches often overlook dynamics induced by
factors such as motion inertia, leading to challenges in scenarios like abrupt
stops after rotation, where the pose remains static while the appearance
changes. This limitation arises from reliance on a single pose as conditional
input, resulting in ambiguity in mapping one pose to multiple appearances. In
this study, we elucidate that variations in human appearance depend not only on
the current frame's pose condition but also on past pose states. Therefore, we
introduce Dyco, a novel method utilizing the delta pose sequence representation
for non-rigid deformations and canonical space to effectively model temporal
appearance variations. To prevent a decrease in the model's generalization
ability to novel poses, we further propose low-dimensional global context to
reduce unnecessary inter-body part dependencies and a quantization operation to
mitigate overfitting of the delta pose sequence by the model. To validate the
effectiveness of our approach, we collected a novel dataset named I3D-Human,
with a focus on capturing temporal changes in clothing appearance under
approximate poses. Through extensive experiments on both I3D-Human and existing
datasets, our approach demonstrates superior qualitative and quantitative
performance. In addition, our inertia-aware 3D human method can unprecedentedly
simulate appearance changes caused by inertia at different velocities.

在这项研究中，我们引入了 Dyco 这一新方法，利用增量姿势序列表示来有效模拟非刚性变形和时间外貌变化，以解决传统方法中姿势与外貌间的歧义性问题。通过在 I3D-Human 和现有数据集上进行广泛实验，我们的方法体现出卓越的质量和数量性能，并能以前所未有的方式模拟不同速度下由惯性引起的外貌变化。

动态环境下的惯性感知三维人体建模与姿势序列

Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose  Sequence

While 3D human body modeling has received much attention in computer vision,
modeling the acoustic equivalent, i.e. modeling 3D spatial audio produced by
body motion and speech, has fallen short in the community. To close this gap,
we present a model that can generate accurate 3D spatial audio for full human
bodies. The system consumes, as input, audio signals from headset microphones
and body pose, and produces, as output, a 3D sound field surrounding the
transmitter's body, from which spatial audio can be rendered at any arbitrary
position in the 3D space. We collect a first-of-its-kind multimodal dataset of
human bodies, recorded with multiple cameras and a spherical array of 345
microphones. In an empirical evaluation, we demonstrate that our model can
produce accurate body-induced sound fields when trained with a suitable loss.
Dataset and code are available online.

通过从头戴式麦克风获取的音频信号和身体姿势作为输入，生成围绕发射者身体的三维声场，从而可以在三维空间的任意位置呈现空间音频。