Large language models (LLMs) frequently hallucinate and produce factual
errors, yet our understanding of why they make these errors remains limited. In
this study, we delve into the underlying mechanisms of LLM hallucinations from
the perspective of inner representations, and discover a salient pattern
associated with hallucinations: correct generations tend to have sharper
context activations in the hidden states of the in-context tokens, compared to
the incorrect ones. Leveraging this insight, we propose an entropy-based metric
to quantify the ``sharpness'' among the in-context hidden states and
incorporate it into the decoding process to formulate a constrained decoding
approach. Experiments on various knowledge-seeking and hallucination benchmarks
demonstrate our approach's consistent effectiveness, for example, achieving up
to an 8.6 point improvement on TruthfulQA. We believe this study can improve
our understanding of hallucinations and serve as a practical solution for
hallucination mitigation.

大型语言模型经常出现虚幻和事实错误，本研究从内部表征的角度探讨了 LLM 幻觉的潜在机制，并发现了与幻觉相关的显著模式：相对于错误生成，正确生成在上下文令牌的隐藏状态中具有更尖锐的上下文激活。基于这一发现，我们提出了一种基于熵的度量方式来量化上下文隐藏状态中的 “尖锐度”，并将其纳入解码过程中，形成一种约束解码方法。在各种知识获取和幻觉评估基准测试上进行的实验表明我们的方法具有持续的有效性，例如在 TruthfulQA 上获得了高达 8.6 个百分点的提升。我们相信这项研究可以改善我们对幻觉的理解，并作为幻觉缓解的实用解决方案。

内部表征视角下的上下文锐度作为警报：减轻幻觉

In-Context Sharpness as Alerts: An Inner Representation Perspective for  Hallucination Mitigation

Current language models decode text token by token according to probabilistic
distribution, and determining the appropriate candidates for the next token is
crucial to ensure generation quality. This study introduces adaptive decoding,
a mechanism that empowers the language models to ascertain a sensible candidate
set during the generation process dynamically. Specifically, we introduce an
entropy-based metric called confidence and conceptualize determining the
optimal candidate set as a confidence-increasing process. The rationality of
including a token in the candidate set is assessed by leveraging the increment
of confidence, enabling the model to determine the most suitable candidate set
adaptively. The experimental results reveal that our method achieves higher
MAUVE and diversity in story generation tasks and maintains certain coherence,
underscoring its superiority over existing algorithms. The code is available at
this https URL

当前语言模型以概率分布逐词解码文本，本研究引入了自适应解码机制，使语言模型能够在生成过程中动态地确定合理的候选集。实验结果显示我们的方法在故事生成任务中实现了更高的 MAUVE 和多样性，同时保持了一定的连贯性，凸现了其优越性。