Tool use is a hallmark of advanced intelligence, exemplified in both animal
behavior and robotic capabilities. This paper investigates the feasibility of
imbuing robots with the ability to creatively use tools in tasks that involve
implicit physical constraints and long-term planning. Leveraging Large Language
Models (LLMs), we develop RoboTool, a system that accepts natural language
instructions and outputs executable code for controlling robots in both
simulated and real-world environments. RoboTool incorporates four pivotal
components: (i) an "Analyzer" that interprets natural language to discern key
task-related concepts, (ii) a "Planner" that generates comprehensive strategies
based on the language input and key concepts, (iii) a "Calculator" that
computes parameters for each skill, and (iv) a "Coder" that translates these
plans into executable Python code. Our results show that RoboTool can not only
comprehend explicit or implicit physical constraints and environmental factors
but also demonstrate creative tool use. Unlike traditional Task and Motion
Planning (TAMP) methods that rely on explicit optimization, our LLM-based
system offers a more flexible, efficient, and user-friendly solution for
complex robotics tasks. Through extensive experiments, we validate that
RoboTool is proficient in handling tasks that would otherwise be infeasible
without the creative use of tools, thereby expanding the capabilities of
robotic systems. Demos are available on our project page:
this https URL

本研究论文通过利用大型语言模型构建了一个名为 RoboTool 的系统，通过接受自然语言指令并输出可执行的机器人控制代码，实现了机器人在包含隐含物理约束和长期规划的任务中的创造性使用工具的能力。通过广泛的实验验证，RoboTool 在处理否则无法完成的任务方面表现出了熟练的水平，从而扩展了机器人系统的能力。