We propose a physics-based method for synthesizing dexterous hand-object
interactions in a full-body setting. While recent advancements have addressed
specific facets of human-object interactions, a comprehensive physics-based
approach remains a challenge. Existing methods often focus on isolated segments
of the interaction process and rely on data-driven techniques that may result
in artifacts. In contrast, our proposed method embraces reinforcement learning
(RL) and physics simulation to mitigate the limitations of data-driven
approaches. Through a hierarchical framework, we first learn skill priors for
both body and hand movements in a decoupled setting. The generic skill priors
learn to decode a latent skill embedding into the motion of the underlying
part. A high-level policy then controls hand-object interactions in these
pretrained latent spaces, guided by task objectives of grasping and 3D target
trajectory following. It is trained using a novel reward function that combines
an adversarial style term with a task reward, encouraging natural motions while
fulfilling the task incentives. Our method successfully accomplishes the
complete interaction task, from approaching an object to grasping and
subsequent manipulation. We compare our approach against kinematics-based
baselines and show that it leads to more physically plausible motions.

我们提出了一种基于物理的方法来合成全身手物交互。通过通过一个层次化框架，并结合强化学习和物理仿真，我们首先在解耦环境中学习身体和手部运动的技能先验。然后，我们使用一个新颖的奖励函数训练一个高层策略，以在预训练的潜在空间中控制手物交互。我们的方法成功完成了从接近物体到抓取和后续操纵的完整交互任务，并展示出比基于运动学的基线方法更具物理合理性的动作。

全身手物互动合成的真实物理模拟

Physically Plausible Full-Body Hand-Object Interaction Synthesis

Existing deep models predict 2D and 3D kinematic poses from video that are
approximately accurate, but contain visible errors that violate physical
constraints, such as feet penetrating the ground and bodies leaning at extreme
angles. In this paper, we present a physics-based method for inferring 3D human
motion from video sequences that takes initial 2D and 3D pose estimates as
input. We first estimate ground contact timings with a novel prediction network
which is trained without hand-labeled data. A physics-based trajectory
optimization then solves for a physically-plausible motion, based on the
inputs. We show this process produces motions that are significantly more
realistic than those from purely kinematic methods, substantially improving
quantitative measures of both kinematic and dynamic plausibility. We
demonstrate our method on character animation and pose estimation tasks on
dynamic motions of dancing and sports with complex contact patterns.

本文介绍了一种基于物理模型的方法，用于从视频中推断 3D 人体运动，该方法利用初始的 2D 和 3D 姿态估计作为输入，并通过物理轨迹优化得出一个符合物理规律的运动。实验结果显示，相较于纯运动学方法，该方法能够显著地提高动作的真实性和准确性。