We present a framework for intuitive robot programming by non-experts,
leveraging natural language prompts and contextual information from the Robot
Operating System (ROS). Our system integrates large language models (LLMs),
enabling non-experts to articulate task requirements to the system through a
chat interface. Key features of the framework include: integration of ROS with
an AI agent connected to a plethora of open-source and commercial LLMs,
automatic extraction of a behavior from the LLM output and execution of ROS
actions/services, support for three behavior modes (sequence, behavior tree,
state machine), imitation learning for adding new robot actions to the library
of possible actions, and LLM reflection via human and environment feedback.
Extensive experiments validate the framework, showcasing robustness,
scalability, and versatility in diverse scenarios, including long-horizon
tasks, tabletop rearrangements, and remote supervisory control. To facilitate
the adoption of our framework and support the reproduction of our results, we
have made our code open-source. You can access it at:
this https URL

通过自然语言提示和来自机器人操作系统（ROS）的上下文信息，我们提出了一个直观的非专业人士机器人编程框架。该框架集成了大型语言模型（LLMs），使非专业人士能够通过聊天界面向系统表达任务要求。它具有与 ROS 集成的 AI 代理与大量开源和商用 LLMs 的连接、从 LLM 输出中提取行为和执行 ROS 动作 / 服务的自动化提取、支持三种行为模式（顺序、行为树、状态机）、模仿学习以将新的机器人行为添加到可能的动作库中以及通过人类和环境反馈实现 LLM 反思的功能。大量实验证实了该框架在各种场景（包括长时间视野任务、桌面重新排列和远程监督控制）中的鲁棒性、可扩展性和多功能性。为了促进我们框架的使用和支持我们结果的可复制性，我们已经将我们的代码开源。您可以在以下网址访问：this https URL