While self-correction has shown promise in improving LLM outputs in terms of
style and quality (e.g. Chen et al., 2023; Madaan et al., 2023), recent
attempts to self-correct logical or reasoning errors often cause correct
answers to become incorrect, resulting in worse performances overall (Huang et
al., 2023). In this paper, we break down the self-correction process into two
core components: mistake finding and output correction. For mistake finding, we
release BIG-Bench Mistake, a dataset of logical mistakes in Chain-of-Thought
reasoning traces. We provide benchmark numbers for several state-of-the-art
LLMs, and demonstrate that LLMs generally struggle with finding logical
mistakes. For output correction, we propose a backtracking method which
provides large improvements when given information on mistake location. We
construe backtracking as a lightweight alternative to reinforcement learning
methods, and show that it remains effective with a reward model at 60-70%
accuracy.

本文将自我纠错过程分解为失误发现和输出校正两个核心组成部分，并对几种最先进的 LLM 进行了基准测试，结果显示 LLM 在发现逻辑错误方面普遍存在困难。针对输出校正，我们提出了一种回溯法，并证明当提供失误位置信息时，该方法能够明显改善性能。我们将回溯法视为轻量级替代强化学习方法的方式，并展示在准确率达到 60-70% 的奖励模型下依然有效。