Large language models (LLMs) recently exhibited remarkable reasoning
capabilities on solving math problems. To further improve this capability, this
work proposes Learning from Mistakes (LeMa), akin to human learning processes.
Consider a human student who failed to solve a math problem, he will learn from
what mistake he has made and how to correct it. Mimicking this error-driven
learning process, LeMa fine-tunes LLMs on mistake-correction data pairs
generated by GPT-4. Specifically, we first collect inaccurate reasoning paths
from various LLMs and then employ GPT-4 as a "corrector" to (1) identify the
mistake step, (2) explain the reason for the mistake, and (3) correct the
mistake and generate the final answer. Experimental results demonstrate the
effectiveness of LeMa: across five backbone LLMs and two mathematical reasoning
tasks, LeMa consistently improves the performance compared with fine-tuning on
CoT data alone. Impressively, LeMa can also benefit specialized LLMs such as
WizardMath and MetaMath, achieving 85.4% pass@1 accuracy on GSM8K and 27.1% on
MATH. This surpasses the SOTA performance achieved by non-execution open-source
models on these challenging tasks. Our code, data and models will be publicly
available at this https URL

利用 Learning from Mistakes（LeMa）方法，本研究利用大型语言模型（LLMs）对数学问题的解决能力进行改进，仿照人类学习过程中的错误驱动学习机制，通过与 GPT-4 生成的错误更正数据对进行微调，实验证明 LeMa 提升了性能，在 5 个 LLMs 和 2 个数学推理任务中持续改善性能，并在特定 LLMs（如 WizardMath 和 MetaMath）上表现出色，在具有挑战性的任务上超越了开源模型的性能。

从错误中学习使 LLM 成为更好的推理者

Learning From Mistakes Makes LLM Better Reasoner

Chain-of-thought (CoT) prompting for language models demonstrates impressive
performance across reasoning tasks, but typically needs labeled exemplars of
the reasoning process. In this work, we introduce a new prompting approach,
Analogical Prompting, designed to automatically guide the reasoning process of
large language models. Inspired by analogical reasoning, a cognitive process in
which humans draw from relevant past experiences to tackle new problems, our
approach prompts language models to self-generate relevant exemplars or
knowledge in the context, before proceeding to solve the given problem. This
method presents several advantages: it obviates the need for labeling or
retrieving exemplars, offering generality and convenience; it can also tailor
the generated exemplars and knowledge to each problem, offering adaptability.
Experimental results show that our approach outperforms 0-shot CoT and manual
few-shot CoT in a variety of reasoning tasks, including math problem solving in
GSM8K and MATH, code generation in Codeforces, and other reasoning tasks in
BIG-Bench.

通过模拟推理为大型语言模型自动生成相关样本或知识的新提示方法在各种推理任务中表现优于零示范的链式思考提示和人工少量示范的链式思考，包括 GSM8K 和 MATH 中的数学问题解决、Codeforces 中的代码生成和 BIG-Bench 中的其他推理任务。

大型语言模型作为类比推理者

Large Language Models as Analogical Reasoners

Chain-of-Thought (CoT) plays a crucial role in reasoning for math problem
solving. We conduct a comprehensive examination of methods for designing CoT,
comparing conventional natural language CoT with various program CoTs,
including the self-describing program, the comment-describing program, and the
non-describing program. Furthermore, we investigate the impact of programming
language on program CoTs, comparing Python and Wolfram Language. Through
extensive experiments on GSM8K, MATHQA, and SVAMP, we find that program CoTs
often have superior effectiveness in math problem solving. Notably, the best
performing combination with 30B parameters beats GPT-3.5-turbo by a significant
margin. The results show that self-describing program offers greater diversity
and thus can generally achieve higher performance. We also find that Python is
a better choice of language than Wolfram for program CoTs. The experimental
results provide a valuable guideline for future CoT designs that take into
account both programming language and coding style for further advancements.
Our datasets and code are publicly available.

针对数学问题求解中的推理，我们对 Chain-of-Thought (CoT) 设计方法进行了全面研究，将常规自然语言 CoT 与各种程序 CoT（包括自说明程序、评论说明程序和非说明程序）进行了比较；此外，我们还研究了编程语言对程序 CoT 的影响，比较了 Python 和 Wolfram Language。通过在 GSM8K、MATHQA 和 SVAMP 上进行的大量实验，我们发现程序 CoT 在数学问题求解中通常具有更好的效果，其中自说明程序以 30B 个参数组合表现最佳，明显超过了 GPT-3.5-turbo。实验结果显示，自说明程序提供了更大的多样性，因此通常可以实现更高的性能。我们还发现，相比于 Wolfram，Python 是程序 CoT 的更好选择的编程语言。实验结果为今后在考虑编程语言和编码风格方面的 CoT 设计提供了有价值的指导。我们的数据集和代码已公开提供。

数学问题解决中的思维链设计

Design of Chain-of-Thought in Math Problem Solving

Many recent developments in large language models focus on prompting them to
perform specific tasks. One effective prompting method is in-context learning,
where the model performs a (possibly new) generation/prediction task given one
(or more) examples. Past work has shown that the choice of examples can make a
large impact on task performance. However, finding good examples is not
straightforward since the definition of a representative group of examples can
vary greatly depending on the task. While there are many existing methods for
selecting in-context examples, they generally score examples independently,
ignoring the dependency between them and the order in which they are provided
to the large language model. In this work, we propose Retrieval for In-Context
Learning (RetICL), a learnable method for modeling and optimally selecting
examples sequentially for in-context learning. We frame the problem of
sequential example selection as a Markov decision process, design an example
retriever model using an LSTM, and train it using proximal policy optimization
(PPO). We validate RetICL on math problem solving datasets and show that it
outperforms both heuristic and learnable baselines, and achieves
state-of-the-art accuracy on the TabMWP dataset. We also use case studies to
show that RetICL implicitly learns representations of math problem solving
strategies.

本研究提出了 Retrieval for In-Context Learning (RetICL)，一种可学习的方法，用于模拟和最佳选择逐个该如何为 in-context learning 选择任务例子。它将顺序示例选择问题作为马尔可夫决策过程，使用 LSTM 设计示例检索器模型，并使用 PPO 进行训练。我们在数学问题求解数据集上验证了 RetICL，表明它优于启发式和可学习的基线，并在 TabMWP 数据集上实现了最先进的准确性。我们还使用案例研究展示了 RetICL 隐含学习了数学问题求解策略的表示方式。