Large language models (LLMs) recently exhibited remarkable reasoning
capabilities on solving math problems. To further improve this capability, this
work proposes Learning from Mistakes (LeMa), akin to human learning processes.
Consider a human student who failed to solve a math problem, he will learn from
what mistake he has made and how to correct it. Mimicking this error-driven
learning process, LeMa fine-tunes LLMs on mistake-correction data pairs
generated by GPT-4. Specifically, we first collect inaccurate reasoning paths
from various LLMs and then employ GPT-4 as a "corrector" to (1) identify the
mistake step, (2) explain the reason for the mistake, and (3) correct the
mistake and generate the final answer. Experimental results demonstrate the
effectiveness of LeMa: across five backbone LLMs and two mathematical reasoning
tasks, LeMa consistently improves the performance compared with fine-tuning on
CoT data alone. Impressively, LeMa can also benefit specialized LLMs such as
WizardMath and MetaMath, achieving 85.4% pass@1 accuracy on GSM8K and 27.1% on
MATH. This surpasses the SOTA performance achieved by non-execution open-source
models on these challenging tasks. Our code, data and models will be publicly
available at this https URL

利用 Learning from Mistakes（LeMa）方法，本研究利用大型语言模型（LLMs）对数学问题的解决能力进行改进，仿照人类学习过程中的错误驱动学习机制，通过与 GPT-4 生成的错误更正数据对进行微调，实验证明 LeMa 提升了性能，在 5 个 LLMs 和 2 个数学推理任务中持续改善性能，并在特定 LLMs（如 WizardMath 和 MetaMath）上表现出色，在具有挑战性的任务上超越了开源模型的性能。

从错误中学习使 LLM 成为更好的推理者

Learning From Mistakes Makes LLM Better Reasoner

Trial-to-trial effects have been found in a number of studies, indicating
that processing a stimulus influences responses in subsequent trials. A special
case are priming effects which have been modelled successfully with
error-driven learning (Marsolek, 2008), implying that participants are
continuously learning during experiments. This study investigates whether
trial-to-trial learning can be detected in an unprimed lexical decision
experiment. We used the Discriminative Lexicon Model (DLM; Baayen et al.,
2019), a model of the mental lexicon with meaning representations from
distributional semantics, which models error-driven incremental learning with
the Widrow-Hoff rule. We used data from the British Lexicon Project (BLP;
Keuleers et al., 2012) and simulated the lexical decision experiment with the
DLM on a trial-by-trial basis for each subject individually. Then, reaction
times were predicted with Generalised Additive Models (GAMs), using measures
derived from the DLM simulations as predictors. We extracted measures from two
simulations per subject (one with learning updates between trials and one
without), and used them as input to two GAMs. Learning-based models showed
better model fit than the non-learning ones for the majority of subjects. Our
measures also provide insights into lexical processing and individual
differences. This demonstrates the potential of the DLM to model behavioural
data and leads to the conclusion that trial-to-trial learning can indeed be
detected in unprimed lexical decision. Our results support the possibility that
our lexical knowledge is subject to continuous changes.

该研究探讨试验中单次学习效应对无提示词汇决策的影响，使用 DLM 模型和 GAM 预测反应时间，发现基于学习的模型在大多数实验中有更好的拟合效果。结果表明，试验中的单次学习可以被探测到，说明我们的词汇知识会不断变化。