When a large language model (LLM) performs complex reasoning by chain of
thought (CoT), it can be highly sensitive to individual mistakes. We have had
to train verifiers to address this issue. As we all know, after human inferring
a conclusion, they often check it by re-verifying it, which can avoid some
mistakes. We propose a new method called self-verification that uses the
conclusion of the CoT as a condition to build a new sample and asks the LLM to
re-predict the original conditions which be masked. We calculate an explainable
verification score based on the accuracy. This method can improve the accuracy
of multiple arithmetics and logical reasoning datasets when using few-shot
learning. we have demonstrated that LLMs can conduct explainable
self-verification of their own conclusions and achieve competitive reasoning
performance. Extensive experimentals have demonstrated that our method can help
multiple large language models with self-verification can avoid interference
from incorrect CoT. Code is available at
https://github.com/WENGSYX/Self-Verification

提出了一种名为自验证的方法，该方法使用推理链的结论作为条件建立新的样本，并要求大型语言模型重新预测原始条件，从而降低了多任务精度误差。经过大量实验验证，此方法可以使大型语言模型避免出现不正确的推理链干扰，并实现具有竞争力的推理性能，可用于算术和逻辑推理数据集的有限次学习。

大型语言模型是具有自我验证功能的推理器

Large Language Models are reasoners with Self-Verification

Pretrained large language models (LLMs) are widely used in many sub-fields of
natural language processing (NLP) and generally known as excellent few-shot
learners with task-specific exemplars. Notably, chain of thought (CoT)
prompting, a recent technique for eliciting complex multi-step reasoning
through step-by-step answer examples, achieved the state-of-the-art
performances in arithmetics and symbolic reasoning, difficult system-2 tasks
that do not follow the standard scaling laws for LLMs. While these successes
are often attributed to LLMs' ability for few-shot learning, we show that LLMs
are decent zero-shot reasoners by simply adding "Let's think step by step"
before each answer. Experimental results demonstrate that our Zero-shot-CoT,
using the same single prompt template, significantly outperforms zero-shot LLM
performances on diverse benchmark reasoning tasks including arithmetics
(MultiArith, GSM8K, AQUA-RAT, SVAMP), symbolic reasoning (Last Letter, Coin
Flip), and other logical reasoning tasks (Date Understanding, Tracking Shuffled
Objects), without any hand-crafted few-shot examples, e.g. increasing the
accuracy on MultiArith from 17.7% to 78.7% and GSM8K from 10.4% to 40.7% with
large InstructGPT model (text-davinci-002), as well as similar magnitudes of
improvements with another off-the-shelf large model, 540B parameter PaLM. The
versatility of this single prompt across very diverse reasoning tasks hints at
untapped and understudied fundamental zero-shot capabilities of LLMs,
suggesting high-level, multi-task broad cognitive capabilities may be extracted
by simple prompting. We hope our work not only serves as the minimal strongest
zero-shot baseline for the challenging reasoning benchmarks, but also
highlights the importance of carefully exploring and analyzing the enormous
zero-shot knowledge hidden inside LLMs before crafting finetuning datasets or
few-shot exemplars.

该研究发现预训练的大型语言模型可以成为良好的零样本推理器，提出了一种零样本的 CoT 方法，将其用于多种复杂的推理任务并取得了显著的性能提升。