Tools are essential for large language models (LLMs) to acquire up-to-date
information and take consequential actions in external environments. Existing
work on tool-augmented LLMs primarily focuses on the broad coverage of tools
and the flexibility of adding new tools. However, a critical aspect that has
surprisingly been understudied is simply how accurately an LLM uses tools for
which it has been trained. We find that existing LLMs, including GPT-4 and
open-source LLMs specifically fine-tuned for tool use, only reach a correctness
rate in the range of 30% to 60%, far from reliable use in practice. We propose
a biologically inspired method for tool-augmented LLMs, simulated trial and
error (STE), that orchestrates three key mechanisms for successful tool use
behaviors in the biological system: trial and error, imagination, and memory.
Specifically, STE leverages an LLM's 'imagination' to simulate plausible
scenarios for using a tool, after which the LLM interacts with the tool to
learn from its execution feedback. Both short-term and long-term memory are
employed to improve the depth and breadth of the exploration, respectively.
Comprehensive experiments on ToolBench show that STE substantially improves
tool learning for LLMs under both in-context learning and fine-tuning settings,
bringing a boost of 46.7% to Mistral-Instruct-7B and enabling it to outperform
GPT-4. We also show effective continual learning of tools via a simple
experience replay strategy.

提出了一种仿生的方法，即模拟试错（STE），通过协调三个关键机制：试错、想象和记忆，为工具增强的大型语言模型（LLMs）提供成功使用工具的行为。通过在 ToolBench 上进行的综合实验，证明 STE 在上下文学习和微调设置下显著改善了 LLMs 的工具学习，为 Mistral-Instruct-7B 带来了 46.7% 的提升，并使其胜过 GPT-4。同时还展示了通过简单的经验重放策略有效地进行工具的持续学习。