Modeling human behaviors in contextual environments has a wide range of
applications in character animation, embodied AI, VR/AR, and robotics. In
real-world scenarios, humans frequently interact with the environment and
manipulate various objects to complete daily tasks. In this work, we study the
problem of full-body human motion synthesis for the manipulation of large-sized
objects. We propose Object MOtion guided human MOtion synthesis (OMOMO), a
conditional diffusion framework that can generate full-body manipulation
behaviors from only the object motion. Since naively applying diffusion models
fails to precisely enforce contact constraints between the hands and the
object, OMOMO learns two separate denoising processes to first predict hand
positions from object motion and subsequently synthesize full-body poses based
on the predicted hand positions. By employing the hand positions as an
intermediate representation between the two denoising processes, we can
explicitly enforce contact constraints, resulting in more physically plausible
manipulation motions. With the learned model, we develop a novel system that
captures full-body human manipulation motions by simply attaching a smartphone
to the object being manipulated. Through extensive experiments, we demonstrate
the effectiveness of our proposed pipeline and its ability to generalize to
unseen objects. Additionally, as high-quality human-object interaction datasets
are scarce, we collect a large-scale dataset consisting of 3D object geometry,
object motion, and human motion. Our dataset contains human-object interaction
motion for 15 objects, with a total duration of approximately 10 hours.

通过将物体运动作为条件，我们提出了一种能够从物体运动中生成全身操作行为的条件扩散框架（OMOMO），该框架学习两个独立的去噪过程，首先从物体运动中预测手的位置，然后基于预测的手位置合成全身姿势，从而显式地强制执行接触约束，从而产生更加真实可信的操作动作。

基于目标运动引导的人类运动合成

Object Motion Guided Human Motion Synthesis

We present a fast and feature-complete differentiable physics engine, Nimble
(nimblephysics.org), that supports Lagrangian dynamics and hard contact
constraints for articulated rigid body simulation. Our differentiable physics
engine offers a complete set of features that are typically only available in
non-differentiable physics simulators commonly used by robotics applications.
We solve contact constraints precisely using linear complementarity problems
(LCPs). We present efficient and novel analytical gradients through the LCP
formulation of inelastic contact that exploit the sparsity of the LCP solution.
We support complex contact geometry, and gradients approximating
continuous-time elastic collision. We also introduce a novel method to compute
complementarity-aware gradients that help downstream optimization tasks avoid
stalling in saddle points. We show that an implementation of this combination
in an existing physics engine (DART) is capable of a 87x single-core speedup
over finite-differencing in computing analytical Jacobians for a single
timestep, while preserving all the expressiveness of original DART.

我们提出了一个快速而完整的可微物理引擎 Nimble，支持拉格朗日动力学和铰接刚体模拟的硬接触约束，并提出了优化下游任务的互补感知梯度的新方法。