Augmenting large language models (LLMs) with external tools has emerged as a
promising approach to extend their utility, empowering them to solve practical
tasks. Existing work typically empowers LLMs as tool users with a manually
designed workflow, where the LLM plans a series of tools in a step-by-step
manner, and sequentially executes each tool to obtain intermediate results
until deriving the final answer. However, they suffer from two challenges in
realistic scenarios: (1) The handcrafted control flow is often ad-hoc and
constraints the LLM to local planning; (2) The LLM is instructed to use only
manually demonstrated tools or well-trained Python functions, which limits its
generalization to new tools. In this work, we first propose Automatic Tool
Chain (ATC), a framework that enables the LLM to act as a multi-tool user,
which directly utilizes a chain of tools through programming. To scale up the
scope of the tools, we next propose a black-box probing method. This further
empowers the LLM as a tool learner that can actively discover and document tool
usages, teaching themselves to properly master new tools. For a comprehensive
evaluation, we build a challenging benchmark named ToolFlow, which diverges
from previous benchmarks by its long-term planning scenarios and complex
toolset. Experiments on both existing datasets and ToolFlow illustrate the
superiority of our framework. Analysis on different settings also validates the
effectiveness and the utility of our black-box probing algorithm.

通过自动工具链和黑盒探测方法，为大型语言模型提供了扩展功能，使其成为多工具用户，能够主动发现和使用新工具，从而解决实际任务中的规划和工具选择的挑战。