Recent language models have a mysterious tendency to generate false but
plausible-sounding text. Such "hallucinations" are an obstacle to the usability
of language-based AI systems and can harm people who rely upon their outputs.
This work shows shows that there is an inherent statistical reason that
pretrained language models hallucinate certain types of facts, having nothing
to do with the transformer LM architecture or data quality. For "arbitrary"
facts whose veracity cannot be determined from the training data, we show that
hallucination is necessary for language models that satisfy a statistical
calibration condition appropriate for generative language models. Specifically,
if the maximum probability of any fact is bounded, we show that the probability
of generating a hallucination is close to the fraction of facts that occur
exactly once in the training data (a "Good-Turing" estimate), even assuming
ideal training data without errors.
One conclusion is that models pretrained to be sufficiently good predictors
(i.e., calibrated) may require post-training to mitigate hallucinations on the
type of arbitrary facts that tend to appear once in the training set. However,
our analysis also suggests that there is no statistical reason that pretraining
will lead to hallucination on facts that tend to appear more than once in the
training data (like references to publications such as articles and books,
whose hallucinations have been particularly notable and problematic) or on
systematic facts (like arithmetic calculations). Therefore, different
architectures and learning algorithms may mitigate these latter types of
hallucinations.

最近的语言模型在生成对外观听起来真实但实际上是虚假的文本方面存在神秘的倾向，这对于基于语言的人工智能系统的可用性构成了障碍，并可能对依赖其输出的人造成伤害。本研究表明，预训练语言模型出现虚构某些类型的事实的统计原因是固有的，与变压器 LM 架构或数据质量无关。对于那些无法从训练数据中确定真实性的 “任意” 事实，我们表明，在适用于生成式语言模型的统计校准条件下，产生幻觉是必要的。具体而言，如果任何事实的最大概率受限，则我们表明产生幻觉的概率接近于训练数据中仅发生一次的事实的分数（一种 “Good-Turing” 估计），即使假设没有错误的理想训练数据。一个结论是，预训练模型要成为足够好的预测器（即校准），可能需要进行后期处理以减轻在训练集中只出现一次的任意事实类型的幻觉。然而，我们的分析还表明，预训练不会在训练数据中多次出现的事实（如对文章和书籍的引用等）或系统性事实（如算术计算）上产生幻觉。因此，不同的架构和学习算法可能会减轻这些后一类幻觉。