Rapid progress in high-level task planning and code generation for open-world
robot manipulation has been witnessed in Embodied AI. However, previous studies
put much effort into general common sense reasoning and task planning
capabilities of large-scale language or multi-modal models, relatively little
effort on ensuring the deployability of generated code on real robots, and
other fundamental components of autonomous robot systems including robot
perception, motion planning, and control. To bridge this ``ideal-to-real'' gap,
this paper presents \textbf{RobotScript}, a platform for 1) a deployable robot
manipulation pipeline powered by code generation; and 2) a code generation
benchmark for robot manipulation tasks in free-form natural language. The
RobotScript platform addresses this gap by emphasizing the unified interface
with both simulation and real robots, based on abstraction from the Robot
Operating System (ROS), ensuring syntax compliance and simulation validation
with Gazebo. We demonstrate the adaptability of our code generation framework
across multiple robot embodiments, including the Franka and UR5 robot arms, and
multiple grippers. Additionally, our benchmark assesses reasoning abilities for
physical space and constraints, highlighting the differences between GPT-3.5,
GPT-4, and Gemini in handling complex physical interactions. Finally, we
present a thorough evaluation on the whole system, exploring how each module in
the pipeline: code generation, perception, motion planning, and even object
geometric properties, impact the overall performance of the system.

RobotScript 是一个平台，用于基于代码生成的部署式机器人操作流程，也是一个自由形式自然语言的机器人操作任务代码生成基准。该平台通过 ROS 的抽象，与仿真和真实机器人实现统一接口，确保语法一致性和与 Gazebo 的仿真验证，并展示了其跨多种机器人实体的代码生成框架的适应性，并评估了 GPT-3.5、GPT-4 和 Gemini 在处理复杂物理交互任务中的推理能力差异，最后进行了全系统的综合评估，探索了代码生成、感知、运动规划和物体几何属性等每个模块对系统整体性能的影响。

RoboScript: 跨真实和模拟环境的自由操作任务的代码生成

RoboScript: Code Generation for Free-Form Manipulation Tasks across Real  and Simulation

Large Language Models (LLMs) have demonstrated remarkable language
understanding and generation capabilities. However, training, deploying, and
accessing these models pose notable challenges, including resource-intensive
demands, extended training durations, and scalability issues. To address these
issues, we introduce a concept of hierarchical, distributed LLM architecture
that aims at enhancing the accessibility and deployability of LLMs across
heterogeneous computing platforms, including general-purpose computers (e.g.,
laptops) and IoT-style devices (e.g., embedded systems). By introducing a
"layered" approach, the proposed architecture enables on-demand accessibility
to LLMs as a customizable service. This approach also ensures optimal
trade-offs between the available computational resources and the user's
application needs. We envision that the concept of hierarchical LLM will
empower extensive, crowd-sourced user bases to harness the capabilities of
LLMs, thereby fostering advancements in AI technology in general.

通过引入层级分布式的大型语言模型（LLM）架构，提高 LLM 在异构计算平台上的可访问性和可部署性，实现按需访问和定制化服务，并在用户与应用需求之间取得最佳权衡，推动人工智能技术的进步。