Understanding how humans cooperatively rearrange household objects is
critical for VR/AR and human-robot interaction. However, in-depth studies on
modeling these behaviors are under-researched due to the lack of relevant
datasets. We fill this gap by presenting CORE4D, a novel large-scale 4D
human-object-human interaction dataset focusing on collaborative object
rearrangement, which encompasses diverse compositions of various object
geometries, collaboration modes, and 3D scenes. With 1K human-object-human
motion sequences captured in the real world, we enrich CORE4D by contributing
an iterative collaboration retargeting strategy to augment motions to a variety
of novel objects. Leveraging this approach, CORE4D comprises a total of 11K
collaboration sequences spanning 3K real and virtual object shapes. Benefiting
from extensive motion patterns provided by CORE4D, we benchmark two tasks
aiming at generating human-object interaction: human-object motion forecasting
and interaction synthesis. Extensive experiments demonstrate the effectiveness
of our collaboration retargeting strategy and indicate that CORE4D has posed
new challenges to existing human-object interaction generation methodologies.
Our dataset and code are available at
this https URL

通过 CORE4D 数据集，我们补充了协作对象重新排列领域的研究空白，这个数据集关注于多种对象几何结构，协作模式和 3D 场景的组合，我们使用迭代协作重新定位策略来扩充模型，共有 11K 协作序列，涵盖 3K 真实和虚拟对象形状。通过 CORE4D 提供的丰富动作模式，我们分别进行人 - 物运动预测和交互合成两个任务的实验，证明了我们的策略有效性并给现有的人 - 物交互生成方法提出了新的挑战。

CORE4D: 一个用于协同物体重新排列的 4D 人体 - 物体 - 人体交互数据集

CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative  Object REarrangement

Can we synthesize 3D humans interacting with scenes without learning from any
3D human-scene interaction data? We propose GenZI, the first zero-shot approach
to generating 3D human-scene interactions. Key to GenZI is our distillation of
interaction priors from large vision-language models (VLMs), which have learned
a rich semantic space of 2D human-scene compositions. Given a natural language
description and a coarse point location of the desired interaction in a 3D
scene, we first leverage VLMs to imagine plausible 2D human interactions
inpainted into multiple rendered views of the scene. We then formulate a robust
iterative optimization to synthesize the pose and shape of a 3D human model in
the scene, guided by consistency with the 2D interaction hypotheses. In
contrast to existing learning-based approaches, GenZI circumvents the
conventional need for captured 3D interaction data, and allows for flexible
control of the 3D interaction synthesis with easy-to-use text prompts.
Extensive experiments show that our zero-shot approach has high flexibility and
generality, making it applicable to diverse scene types, including both indoor
and outdoor environments.

可以不借助任何 3D 人 - 场景交互数据合成 3D 人与场景交互吗？我们提出了 GenZI，这是第一个零样本方法，用于生成 3D 人与场景的交互。GenZI 的关键在于从大型视觉 - 语言模型 (VLMs) 中提取交互先验知识，这些先验知识学习了丰富的二维人 - 场景组合的语义空间。通过给定自然语言描述和 3D 场景中所需交互的粗略点位置，我们首先利用 VLMs 来想象描绘在场景的多个渲染视图中的可信的二维人交互。然后，我们通过与 2D 交互假设的一致性引导，制定一个鲁棒的迭代优化过程，合成场景中的 3D 人模型的姿态和形状。与现有的基于学习的方法相比，GenZI 避免了传统上对捕获的 3D 交互数据的需求，并允许使用简单易用的文字提示对 3D 交互合成进行灵活控制。大量实验证明我们的零样本方法具有高灵活性和广泛适用性，可适用于包括室内和室外环境在内的各种场景类型。