Enabling robots to autonomously perform hybrid motions in diverse
environments can be beneficial for long-horizon tasks such as material
handling, household chores, and work assistance. This requires extensive
exploitation of intrinsic motion capabilities, extraction of affordances from
rich environmental information, and planning of physical interaction behaviors.
Despite recent progress has demonstrated impressive humanoid whole-body control
abilities, they struggle to achieve versatility and adaptability for new tasks.
In this work, we propose HYPERmotion, a framework that learns, selects and
plans behaviors based on tasks in different scenarios. We combine reinforcement
learning with whole-body optimization to generate motion for 38 actuated joints
and create a motion library to store the learned skills. We apply the planning
and reasoning features of the large language models (LLMs) to complex
loco-manipulation tasks, constructing a hierarchical task graph that comprises
a series of primitive behaviors to bridge lower-level execution with
higher-level planning. By leveraging the interaction of distilled spatial
geometry and 2D observation with a visual language model (VLM) to ground
knowledge into a robotic morphology selector to choose appropriate actions in
single- or dual-arm, legged or wheeled locomotion. Experiments in simulation
and real-world show that learned motions can efficiently adapt to new tasks,
demonstrating high autonomy from free-text commands in unstructured scenes.
Videos and website: hy-motion.github.io/

提出了一种基于 HYPERmotion 框架的自主学习与行为规划方法，该方法结合了强化学习和全身优化，并且利用复杂的环境信息和大型语言模型的规划和推理功能，可以使机器人实现高自由度的行为适应性。