Although chain-of-thought (CoT) prompting combined with language models has
achieved encouraging results on complex reasoning tasks, the naive greedy
decoding used in CoT prompting usually causes the repetitiveness and local
optimality. To address this shortcoming, ensemble-optimization tries to obtain
multiple reasoning paths to get the final answer assembly. However, current
ensemble-optimization methods either simply employ rule-based post-processing
such as \textit{self-consistency}, or train an additional model based on
several task-related human annotations to select the best one among multiple
reasoning paths, yet fail to generalize to realistic settings where the type of
input questions is unknown or the answer format of reasoning paths is unknown.
To avoid their limitations, we propose \textbf{self-agreement}, a generalizable
ensemble-optimization method applying in almost all scenarios where the type of
input questions and the answer format of reasoning paths may be known or
unknown. Self-agreement firstly samples from language model's decoder to
generate a \textit{diverse} set of reasoning paths, and subsequently prompts
the language model \textit{one more time} to determine the optimal answer by
selecting the most \textit{agreed} answer among the sampled reasoning paths.
Self-agreement simultaneously achieves remarkable performance on six public
reasoning benchmarks and superior generalization capabilities.

通过自一致性和自一致性来解决链式思考提示中的贪婪解码的问题，并提出了自一致性的方法，它能够在几乎所有场景中得到优化，同时在六个公开推理基准中表现出显著的性能和卓越的普适性。

再问一次：自我一致性提升（几乎）所有情境下的语言模型推理能力

Ask One More Time: Self-Agreement Improves Reasoning of Language Models  in (Almost) All Scenarios

Finding an agreement among diverse opinions is a challenging topic in
multiagent systems. Recently, large language models (LLMs) have shown great
potential in addressing this challenge due to their remarkable capabilities in
comprehending human opinions and generating human-like text. However, they
typically rely on extensive human-annotated data. In this paper, we propose
Self-Agreement, a novel framework for fine-tuning LLMs to autonomously find
agreement using data generated by LLM itself. Specifically, our approach
employs the generative pre-trained transformer-3 (GPT-3) to generate multiple
opinions for each question in a question dataset and create several agreement
candidates among these opinions. Then, a bidirectional encoder representations
from transformers (BERT)-based model evaluates the agreement score of each
agreement candidate and selects the one with the highest agreement score. This
process yields a dataset of question-opinion-agreements, which we use to
fine-tune a pre-trained LLM for discovering agreements among diverse opinions.
Remarkably, a pre-trained LLM fine-tuned by our Self-Agreement framework
achieves comparable performance to GPT-3 with only 1/25 of its parameters,
showcasing its ability to identify agreement among various opinions without the
need for human-annotated data.

本文提出了一种名为 Self-Agreement 的新框架，该框架利用由大型语言模型生成的数据来自主找到多种意见之间的一致性，通过 fine-tuning 模型，比 GPT-3 具有更高的识别多种意见之间一致性的性能，并且只需要 GPT-3 的 1/25 的参数。