Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools (e.g., computation libraries and symbolic solvers), thereby amalgamating the analytical prowess of language and the computational efficiency of tools. To train ToRA, we curate interactive tool-use trajectories on mathematical datasets, apply imitation learning on the annotations, and propose output space shaping to further refine models' reasoning behavior. As a result, ToRA models significantly outperform open-source models on 10 mathematical reasoning datasets across all scales with 13%-19% absolute improvements on average. Notably, ToRA-7B reaches 44.6% on the competition-level dataset MATH, surpassing the best open-source model WizardMath-70B by 22% absolute. ToRA-34B is also the first open-source model that achieves an accuracy exceeding 50% on MATH, which significantly outperforms GPT-4's CoT result, and is competitive with GPT-4 solving problems with programs. Additionally, we conduct a comprehensive analysis of the benefits and remaining challenges of tool interaction for mathematical reasoning, providing valuable insights for future research.

通过无缝集成自然语言推理与外部工具的使用，我们提出了一系列名为ToRA的工具集成推理代理，旨在解决复杂的数学问题，从而融合了语言的分析能力和工具的计算效率。通过在数学数据集上训练ToRA，我们验证其在10个数学推理数据集上显著优于开源模型，取得了13%至19%的绝对改进。尤其是，在MATH竞赛级数据集上，ToRA-7B达到了44.6%，超过最好的开源模型WizardMath-70B 22%。此外，我们还对工具交互在数学推理中的益处和挑战进行了全面分析，为未来的研究提供了有价值的见解。

ToRA数学问题解决的工具整合推理代理