We investigate the internal behavior of Transformer-based Large Language
Models (LLMs) when they generate factually incorrect text. We propose modeling
factual queries as Constraint Satisfaction Problems and use this framework to
investigate how the model interacts internally with factual constraints.
Specifically, we discover a strong positive relation between the model's
attention to constraint tokens and the factual accuracy of its responses. In
our curated suite of 11 datasets with over 40,000 prompts, we study the task of
predicting factual errors with the Llama-2 family across all scales (7B, 13B,
70B). We propose SAT Probe, a method probing self-attention patterns, that can
predict constraint satisfaction and factual errors, and allows early error
identification. The approach and findings demonstrate how using the mechanistic
understanding of factuality in LLMs can enhance reliability.

使用 Transformer-based 大型语言模型，研究当其生成错误的文本时的内部行为；将事实查询建模为约束满足问题，并使用该框架来研究模型与事实约束的内部交互；通过研究包含超过 40,000 个提示的 11 个数据集，在各种规模的 Llama-2 系列（7B、13B、70B）上研究预测事实错误的任务；提出了一种可以预测约束满足和事实错误、并且可以早期发现错误的 self-attention 模式探查方法 SAT Probe；这些方法和发现表明，利用对大型语言模型中事实性的机械理解可以提高可靠性。