Synthesizing physically plausible human motions in 3D scenes is a challenging
problem. Kinematics-based methods cannot avoid inherent artifacts (e.g.,
penetration and foot skating) due to the lack of physical constraints.
Meanwhile, existing physics-based methods cannot generalize to multi-object
scenarios since the policy trained with reinforcement learning has limited
modeling capacity. In this work, we present a framework that enables physically
simulated characters to perform long-term interaction tasks in diverse,
cluttered, and unseen scenes. The key idea is to decompose human-scene
interactions into two fundamental processes, Interacting and Navigating, which
motivates us to construct two reusable Controller, i.e., InterCon and NavCon.
Specifically, InterCon contains two complementary policies that enable
characters to enter and leave the interacting state (e.g., sitting on a chair
and getting up). To generate interaction with objects at different places, we
further design NavCon, a trajectory following policy, to keep characters'
locomotion in the free space of 3D scenes. Benefiting from the divide and
conquer strategy, we can train the policies in simple environments and
generalize to complex multi-object scenes. Experimental results demonstrate
that our framework can synthesize physically plausible long-term human motions
in complex 3D scenes. Code will be publicly released at
this https URL

构建了一个框架，通过分解人 - 场景交互为相互作用和导航两个基本过程，实现了在复杂的三维场景中进行长期交互任务的真实物理模拟角色。

在 3D 场景中合成真实可行的人类动作

Synthesizing Physically Plausible Human Motions in 3D Scenes

3D human pose estimation from a monocular video has recently seen significant
improvements. However, most state-of-the-art methods are kinematics-based,
which are prone to physically implausible motions with pronounced artifacts.
Current dynamics-based methods can predict physically plausible motion but are
restricted to simple scenarios with static camera view. In this work, we
present D&D (Learning Human Dynamics from Dynamic Camera), which leverages the
laws of physics to reconstruct 3D human motion from the in-the-wild videos with
a moving camera. D&D introduces inertial force control (IFC) to explain the 3D
human motion in the non-inertial local frame by considering the inertial forces
of the dynamic camera. To learn the ground contact with limited annotations, we
develop probabilistic contact torque (PCT), which is computed by differentiable
sampling from contact probabilities and used to generate motions. The contact
state can be weakly supervised by encouraging the model to generate correct
motions. Furthermore, we propose an attentive PD controller that adjusts target
pose states using temporal information to obtain smooth and accurate pose
control. Our approach is entirely neural-based and runs without offline
optimization or simulation in physics engines. Experiments on large-scale 3D
human motion benchmarks demonstrate the effectiveness of D&D, where we exhibit
superior performance against both state-of-the-art kinematics-based and
dynamics-based methods. Code is available at this https URL

本研究提出了一种利用 D&D 学习人体动力学的方法来实现从非惯性的本地帧考虑动态摄像机的惯性力控制来重构 3D 人体运动，并通过概率接触扭矩和基于注意力的 PD 控制来实现弱监督的接触状态，从而在大规模 3D 人体运动基准测试中表现出优越的性能。