Large Language Model (LLM) agents significantly extend the capabilities of
standalone LLMs, empowering them to interact with external tools (e.g., APIs,
functions) and complete complex tasks in a self-directed fashion. The challenge
of tool use demands that LLMs not only understand user queries and generate
answers but also excel in task planning, memory management, tool invocation,
and result summarization. While traditional approaches focus on training a
single LLM with all these capabilities, performance limitations become
apparent, particularly with smaller models. Moreover, the entire LLM may
require retraining when tools are updated. To overcome these challenges, we
propose a novel strategy that decomposes the aforementioned capabilities into a
planner, caller, and summarizer. Each component is implemented by a single LLM
that focuses on a specific capability and collaborates with other components to
accomplish the task. This modular framework facilitates individual updates and
the potential use of smaller LLMs for building each capability. To effectively
train this framework, we introduce a two-stage training paradigm. First, we
fine-tune a backbone LLM on the entire dataset without discriminating
sub-tasks, providing the model with a comprehensive understanding of the task.
Second, the fine-tuned LLM is used to instantiate the planner, caller, and
summarizer respectively, which are continually fine-tuned on respective
sub-tasks. Evaluation across various tool-use benchmarks illustrates that our
proposed multi-LLM framework surpasses the traditional single-LLM approach,
highlighting its efficacy and advantages in tool learning.

我们提出了一个模块化的多语言模型框架，将大型语言模型能力分解为规划器、调用器和摘要生成器，并通过两阶段训练范式有效地训练该框架，该框架在各种工具使用基准测试中表现出超越传统单语言模型方法的效果，凸显了其在工具学习中的功效和优势。