This paper introduces the MCT Self-Refine (MCTSr) algorithm, an innovative
integration of Large Language Models (LLMs) with Monte Carlo Tree Search
(MCTS), designed to enhance performance in complex mathematical reasoning
tasks. Addressing the challenges of accuracy and reliability in LLMs,
particularly in strategic and mathematical reasoning, MCTSr leverages
systematic exploration and heuristic self-refine mechanisms to improve
decision-making frameworks within LLMs. The algorithm constructs a Monte Carlo
search tree through iterative processes of Selection, self-refine,
self-evaluation, and Backpropagation, utilizing an improved Upper Confidence
Bound (UCB) formula to optimize the exploration-exploitation balance. Extensive
experiments demonstrate MCTSr's efficacy in solving Olympiad-level mathematical
problems, significantly improving success rates across multiple datasets,
including GSM8K, GSM Hard, MATH, and Olympiad-level benchmarks, including Math
Odyssey, AIME, and OlympiadBench. The study advances the application of LLMs in
complex reasoning tasks and sets a foundation for future AI integration,
enhancing decision-making accuracy and reliability in LLM-driven applications.

这篇论文介绍了 MCT Self-Refine (MCTSr) 算法，它是大型语言模型（LLM）与蒙特卡洛树搜索（MCTS）的创新集成，在复杂的数学推理任务中提高性能。通过解决 LLM 在战略和数学推理中的准确性和可靠性挑战，MCTSr 利用系统化的探索和启发式自我精炼机制来改进 LLM 内的决策框架。该算法通过选择、自我精炼、自我评估和反向传播的迭代过程构建蒙特卡洛搜索树，利用改进的 Upper Confidence Bound（UCB）公式优化探索 - 利用平衡。广泛的实验证明了 MCTSr 在解决奥林匹克数学问题方面的效果，显著提高了多个数据集的成功率，包括 GSM8K、GSM Hard、MATH 以及 Math Odyssey、AIME 和 OlympiadBench 等奥林匹克级别基准。该研究推动了 LLM 在复杂推理任务中的应用，为未来的人工智能集成奠定了基础，提高了 LLM 驱动应用中的决策准确性和可靠性。