Large language models (LLMs) that do not give consistent answers across
contexts are problematic when used for tasks with expectations of consistency,
e.g., question-answering, explanations, etc. Our work presents an evaluation
benchmark for self-consistency in cases of under-specification where two or
more answers can be correct. We conduct a series of behavioral experiments on
the OpenAI model suite using an ambiguous integer sequence completion task. We
find that average consistency ranges from 67\% to 82\%, far higher than would
be predicted if a model's consistency was random, and increases as model
capability improves. Furthermore, we show that models tend to maintain
self-consistency across a series of robustness checks, including prompting
speaker changes and sequence length changes. These results suggest that
self-consistency arises as an emergent capability without specifically training
for it. Despite this, we find that models are uncalibrated when judging their
own consistency, with models displaying both over- and under-confidence. We
also propose a nonparametric test for determining from token output
distribution whether a model assigns non-trivial probability to alternative
answers. Using this test, we find that despite increases in self-consistency,
models usually place significant weight on alternative, inconsistent answers.
This distribution of probability mass provides evidence that even highly
self-consistent models internally compute multiple possible responses.

在含糊的整数序列填充任务中，我们在 OpenAI 模型套件上进行了一系列行为实验，发现平均一致性介于 67％和 82％之间，远高于模型一致性随机的预测，并随着模型能力的提高而增加。此外，我们发现模型在自我一致性上具有不良标定性，并且通常倾向于在潜在的答案中分配显著的权重。