Large language models (LLMs) are susceptible to hallucination, which sparked
a widespread effort to detect and prevent them. Recent work attempts to
mitigate hallucinations by intervening in the model's computation during
generation, using different setups and heuristics. Those works lack separation
between different hallucination causes. In this work, we first introduce an
approach for constructing datasets based on the model knowledge for detection
and intervention methods in closed-book and open-book question-answering
settings. We then characterize the effect of different choices for
intervention, such as the intervened components (MLPs, attention block,
residual stream, and specific heads), and how often and how strongly to
intervene. We find that intervention success varies depending on the component,
with some components being detrimental to language modeling capabilities.
Finally, we find that interventions can benefit from pre-hallucination steering
direction instead of post-hallucination. The code is available at
this https URL

大语言模型容易产生幻觉，本文提出了一种基于模型知识构建数据集的方法，用于在封闭式和开放式问答环境中检测和干预。我们对干预的不同选择，如干预组件（多层感知机，注意力模块，残差流和特定头部），干预的频率和强度进行了特征化，并发现干预的成功与组件相关，某些组件对语言建模能力有害。最后，我们发现干预可以从产生幻觉之前的引导方向中受益。代码可在此 URL 获得。

构建对抗 LLMs 幻觉的基准和干预措施

Constructing Benchmarks and Interventions for Combating Hallucinations  in LLMs

Recent advances in open-domain question answering (ODQA) have demonstrated
impressive accuracy on standard Wikipedia style benchmarks. However, it is less
clear how robust these models are and how well they perform when applied to
real-world applications in drastically different domains. While there has been
some work investigating how well ODQA models perform when tested for
out-of-domain (OOD) generalization, these studies have been conducted only
under conservative shifts in data distribution and typically focus on a single
component (ie. retrieval) rather than an end-to-end system. In response, we
propose a more realistic and challenging domain shift evaluation setting and,
through extensive experiments, study end-to-end model performance. We find that
not only do models fail to generalize, but high retrieval scores often still
yield poor answer prediction accuracy. We then categorize different types of
shifts and propose techniques that, when presented with a new dataset, predict
if intervention methods are likely to be successful. Finally, using insights
from this analysis, we propose and evaluate several intervention methods which
improve end-to-end answer F1 score by up to 24 points.

研究开放领域问答（ODQA）中，当应用于广泛不同的领域时，此类模型的稳健性和应用性能。该研究提出了一个更现实和具有挑战性的领域转移评估环境，并研究了端到端的模型性能。他们发现，不仅模型在推广方面表现出失败，且高检索分数通常也不能提供准确的答案预测。最后，研究提出并评估了多种干预方法，其提高了端到端答案 F1 得分，最多可达 24 个点。