Many of the recent capabilities demonstrated by Large Language Models (LLMs)
arise primarily from their ability to exploit contextual information. In this
paper, we explore ways to improve reasoning capabilities of LLMs through (1)
exploration of different chains of thought and (2) validation of the individual
steps of the reasoning process. We propose three general principles that a
model should adhere to while reasoning: (i) Relevance, (ii) Mathematical
Accuracy, and (iii) Logical Consistency. We apply these constraints to the
reasoning steps generated by the LLM to improve the accuracy of the final
generation. The constraints are applied in the form of verifiers: the model
itself is asked to verify if the generated steps satisfy each constraint. To
further steer the generations towards high-quality solutions, we use the
perplexity of the reasoning steps as an additional verifier. We evaluate our
method on 4 distinct types of reasoning tasks, spanning a total of 9 different
datasets. Experiments show that our method is always better than vanilla
generation, and, in 6 out of the 9 datasets, it is better than best-of N
sampling which samples N reasoning chains and picks the lowest perplexity
generation.

通过探索不同的思维链和验证推理过程中的各个步骤，我们提出了三个模型应遵循的原则（相关性、数学准确性和逻辑一致性），并将这些原则应用于大型语言模型的推理步骤，以提高最终生成结果的准确性。通过使用困惑度作为额外的验证器来引导高质量解决方案的生成，我们在 4 种不同类型的推理任务上评估了我们的方法，涵盖了共计 9 个不同的数据集。实验证明，我们的方法始终优于基准生成，并且在 9 个数据集中的 6 个数据集中，优于最佳的 N 个采样方法。

链式思维引导的通用验证

General Purpose Verification for Chain of Thought Prompting

State-of-the-art language models can match human performance on many tasks,
but they still struggle to robustly perform multi-step mathematical reasoning.
To diagnose the failures of current models and support research, we introduce
GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math
word problems. We find that even the largest transformer models fail to achieve
high test performance, despite the conceptual simplicity of this problem
distribution. To increase performance, we propose training verifiers to judge
the correctness of model completions. At test time, we generate many candidate
solutions and select the one ranked highest by the verifier. We demonstrate
that verification significantly improves performance on GSM8K, and we provide
strong empirical evidence that verification scales more effectively with
increased data than a finetuning baseline.

介绍一个 8.5K 高质量的语言多样的小学数学问题数据集 GSM8K，表明即便是最大的变压器模型仍然无法在多步数学推理方面实现高的测试性能，提出使用 verifiers 验证模型的正确性并通过排名选择最优解来提高性能。