Despite tremendous advancements in large language models (LLMs) over recent
years, a notably urgent challenge for their practical deployment is the
phenomenon of hallucination, where the model fabricates facts and produces
non-factual statements. In response, we propose PoLLMgraph, a Polygraph for
LLMs, as an effective model-based white-box detection and forecasting approach.
PoLLMgraph distinctly differs from the large body of existing research that
concentrates on addressing such challenges through black-box evaluations. In
particular, we demonstrate that hallucination can be effectively detected by
analyzing the LLM's internal state transition dynamics during generation via
tractable probabilistic models. Experimental results on various open-source
LLMs confirm the efficacy of PoLLMgraph, outperforming state-of-the-art methods
by a considerable margin, evidenced by over 20% improvement in AUC-ROC on
common benchmarking datasets like TruthfulQA. Our work paves a new way for
model-based white-box analysis of LLMs, motivating the research community to
further explore, understand, and refine the intricate dynamics of LLM
behaviors.

我们提出了 PoLLMgraph，一种用于大型语言模型的多种同类白盒检测和预测方法，通过分析生成过程中 LLM 的内部状态转换动态来有效地检测虚构现象，实验证明其比现有方法在 TruthfulQA 等常见基准数据集上具有 20% 以上的 AUC-ROC 改进效果，为 LLM 的模型化白盒分析开辟了新的途径。