Humanoid robots, with their human-like embodiment, have the potential to integrate seamlessly into human environments. Critical to their coexistence and cooperation with humans is the ability to understand natural language communications and exhibit human-like behaviors. This work focuses on generating diverse whole-body motions for humanoid robots from language descriptions. We leverage human motion priors from extensive human motion datasets to initialize humanoid motions and employ the commonsense reasoning capabilities of Vision Language Models (VLMs) to edit and refine these motions. Our approach demonstrates the capability to produce natural, expressive, and text-aligned humanoid motions, validated through both simulated and real-world experiments. More videos can be found at https://ut-austin-rpl.github.io/Harmon/.

本研究解决了类人机器人在与人类环境的共存中对自然语言理解和人类行为表现的能力不足的问题。通过利用人类运动数据库中的运动先验知识，并结合视觉语言模型的常识推理能力，我们提出了一种生成多样化、自然表达且与文本对齐的类人机器人全身运动的方法。实验结果表明，该方法可以有效提升类人机器人在真实世界中的运动表现。

Harmon: 从语言描述生成类人智能机器人的全身运动