We propose CG-HOI, the first method to address the task of generating dynamic
3D human-object interactions (HOIs) from text. We model the motion of both
human and object in an interdependent fashion, as semantically rich human
motion rarely happens in isolation without any interactions. Our key insight is
that explicitly modeling contact between the human body surface and object
geometry can be used as strong proxy guidance, both during training and
inference. Using this guidance to bridge human and object motion enables
generating more realistic and physically plausible interaction sequences, where
the human body and corresponding object move in a coherent manner. Our method
first learns to model human motion, object motion, and contact in a joint
diffusion process, inter-correlated through cross-attention. We then leverage
this learned contact for guidance during inference synthesis of realistic,
coherent HOIs. Extensive evaluation shows that our joint contact-based
human-object interaction approach generates realistic and physically plausible
sequences, and we show two applications highlighting the capabilities of our
method. Conditioned on a given object trajectory, we can generate the
corresponding human motion without re-training, demonstrating strong
human-object interdependency learning. Our approach is also flexible, and can
be applied to static real-world 3D scene scans.

我们提出了 CG-HOI，这是第一个解决从文本中生成动态 3D 人物 - 物体交互（HOIs）任务的方法。我们通过在训练和推断过程中显式地建模人体表面与物体几何之间的接触来引导人体和物体的运动，从而生成更真实、物理上可行的交互序列。