Self-consistency with chain-of-thought prompting (CoT) has demonstrated
remarkable performance gains on various challenging tasks, by utilizing
multiple reasoning paths sampled from large language models (LLMs). However,
self-consistency relies on the answer extraction process to aggregate multiple
solutions, which is not applicable to free-form answers. In this work, we
propose Universal Self-Consistency (USC), which leverages LLMs themselves to
select the most consistent answer among multiple candidates. We evaluate USC on
a variety of benchmarks, including mathematical reasoning, code generation,
long-context summarization, and open-ended question answering. On open-ended
generation tasks where the original self-consistency method is not applicable,
USC effectively utilizes multiple samples and improves the performance. For
mathematical reasoning, USC matches the standard self-consistency performance
without requiring the answer formats to be similar. Finally, without access to
execution results, USC also matches the execution-based voting performance on
code generation.

利用大型语言模型采样的多个推理路径，结合自我一致性和链式思维提示，在各种具有挑战性的任务上取得了显著的性能提升。本研究提出了一种通用的自我一致性方法（USC），它利用语言模型自身从多个候选答案中选择最一致的答案。我们在包括数学推理、代码生成、长篇摘要和开放式问答在内的多个数据集上评估了 USC 的性能。在原来的自我一致性方法不适用的开放式生成任务中，USC 有效利用多个样本并提高了性能。对于数学推理，USC 在不要求答案格式相似的情况下，达到了标准自我一致性的性能。最后，在无法访问执行结果的情况下，USC 也达到了代码生成中基于执行的投票性能。