Interacting with a software system via a chatbot can be challenging, especially when the chatbot needs to generate API calls, in the right order and with the right parameters, to communicate with the system. API calling in chatbot systems poses significant challenges, particularly in complex, multi-step tasks requiring accurate API selection and execution. We contribute to this domain in three ways: first, by introducing a novel dataset designed to assess models on API function selection, parameter generation, and nested API calls; second, by benchmarking state-of-the-art language models across varying levels of complexity to evaluate their performance in API function generation and parameter accuracy; and third, by proposing an enhanced API routing method that combines general-purpose large language models for API selection with fine-tuned models for parameter generation and some prompt engineering approach. These approaches lead to substantial improvements in handling complex API tasks, offering practical advancements for real-world API-driven chatbot systems.

本研究针对聊天机器人与软件系统交互中的API调用问题，提出了一种新数据集以评估模型在API函数选择、参数生成和嵌套API调用方面的表现。通过测试最新语言模型的性能并引入改进的API路由方法，本工作显著提升了复杂API任务的处理能力，促进了实际API驱动聊天系统的进展。

CallNavi: 大型语言模型中的函数调用路由和调用研究与挑战