In this study, we tackle the complex task of generating 3D human-object
interactions (HOI) from textual descriptions in a zero-shot text-to-3D manner.
We identify and address two key challenges: the unsatisfactory outcomes of
direct text-to-3D methods in HOI, largely due to the lack of paired
text-interaction data, and the inherent difficulties in simultaneously
generating multiple concepts with complex spatial relationships. To effectively
address these issues, we present InterFusion, a two-stage framework
specifically designed for HOI generation. InterFusion involves human pose
estimations derived from text as geometric priors, which simplifies the
text-to-3D conversion process and introduces additional constraints for
accurate object generation. At the first stage, InterFusion extracts 3D human
poses from a synthesized image dataset depicting a wide range of interactions,
subsequently mapping these poses to interaction descriptions. The second stage
of InterFusion capitalizes on the latest developments in text-to-3D generation,
enabling the production of realistic and high-quality 3D HOI scenes. This is
achieved through a local-global optimization process, where the generation of
human body and object is optimized separately, and jointly refined with a
global optimization of the entire scene, ensuring a seamless and contextually
coherent integration. Our experimental results affirm that InterFusion
significantly outperforms existing state-of-the-art methods in 3D HOI
generation.

本研究利用 InterFusion 框架，通过从文本中提取到的人体姿势估计作为几何先验，实现了具有复杂空间关系的多个概念的高质量三维人物 - 物体交互场景的生成，并且结果表明 InterFusion 在三维 HOI 生成方面显著优于现有的最先进方法。