Real-world long-horizon embodied planning underpins embodied AI. To accomplish long-horizon tasks, agents need to decompose abstract instructions into detailed steps. Prior works mostly rely on GPT-4V for task decomposition into predefined actions, which limits task diversity due to GPT-4V's finite understanding of larger skillsets. Therefore, we present ReLEP, a groundbreaking framework for Real world Long-horizon Embodied Planning, which can accomplish a wide range of daily tasks. At its core lies a fine-tuned large vision language model that formulates plans as sequences of skill functions according to input instruction and scene image. These functions are selected from a carefully designed skill library. ReLEP is also equipped with a Memory module for plan and status recall, and a Robot Configuration module for versatility across robot types. In addition, we propose a semi-automatic data generation pipeline to tackle dataset scarcity. Real-world off-line experiments across eight daily embodied tasks demonstrate that ReLEP is able to accomplish long-horizon embodied tasks and outperforms other state-of-the-art baseline methods.

本研究解决了现有嵌入式人工智能在长远任务规划中的局限，提出了一种新颖的ReLEP框架，该框架能够将抽象指令细化为详细步骤并完成多样化的日常任务。通过使用经过微调的大型视觉语言模型和精心设计的技能库，ReLEP展示了在真实环境中执行长远嵌入式任务的能力，并且在与现有的最先进方法的比较中表现出更优的性能。

ReLEP：一种新的真实场景长远嵌入式规划框架