This work is about estimating the hallucination rate for in-context learning
(ICL) with Generative AI. In ICL, a conditional generative model (CGM) is
prompted with a dataset and asked to make a prediction based on that dataset.
The Bayesian interpretation of ICL assumes that the CGM is calculating a
posterior predictive distribution over an unknown Bayesian model of a latent
parameter and data. With this perspective, we define a \textit{hallucination}
as a generated prediction that has low-probability under the true latent
parameter. We develop a new method that takes an ICL problem -- that is, a CGM,
a dataset, and a prediction question -- and estimates the probability that a
CGM will generate a hallucination. Our method only requires generating queries
and responses from the model and evaluating its response log probability. We
empirically evaluate our method on synthetic regression and natural language
ICL tasks using large language models.

这项工作是关于使用生成性人工智能估计上下文学习（ICL）的幻觉率。我们开发了一种新方法，用于估计条件生成模型在 ICL 问题上生成幻觉的概率。我们在合成回归和自然语言 ICL 任务上对我们的方法进行了实证评估。

生成 AI 虚构率估计

Estimating the Hallucination Rate of Generative AI

We develop a principled procedure for determining when a large language model
(LLM) should abstain from responding (e.g., by saying "I don't know") in a
general domain, instead of resorting to possibly "hallucinating" a non-sensical
or incorrect answer. Building on earlier approaches that use self-consistency
as a more reliable measure of model confidence, we propose using the LLM itself
to self-evaluate the similarity between each of its sampled responses for a
given query. We then further leverage conformal prediction techniques to
develop an abstention procedure that benefits from rigorous theoretical
guarantees on the hallucination rate (error rate). Experimentally, our
resulting conformal abstention method reliably bounds the hallucination rate on
various closed-book, open-domain generative question answering datasets, while
also maintaining a significantly less conservative abstention rate on a dataset
with long responses (Temporal Sequences) compared to baselines using
log-probability scores to quantify uncertainty, while achieveing comparable
performance on a dataset with short answers (TriviaQA). To evaluate the
experiments automatically, one needs to determine if two responses are
equivalent given a question. Following standard practice, we use a thresholded
similarity function to determine if two responses match, but also provide a
method for calibrating the threshold based on conformal prediction, with
theoretical guarantees on the accuracy of the match prediction, which might be
of independent interest.

我们开发了一个原则性的方法，确定一个大型语言模型在一般领域中，何时应该放弃回答而选择回答 “我不知道”，而不是胡言乱语或错误答案。通过使用自洽性作为一种更可靠的模型置信度度量的早期方法的基础上，我们提出使用语言模型本身来自我评估其对于给定查询的每个采样响应之间的相似性。然后，我们进一步利用符合预测技术来开发一种放弃回答的方法，该方法在误报率（错误率）上具有严格的理论保证。在实验中，我们的符合预测放弃方法可可靠地限制了各种闭书、开放领域的生成问答数据集的胡言乱语率，同时与基于对数概率分数量化不确定性的基线相比，在长答案的数据集（时间序列）上保持了一个显著较低的放弃率，同时在短答案的数据集（TriviaQA）上实现了可比较的性能。为了自动评估实验，需要确定两个答案在给定问题下是否等价。遵循标准做法，我们使用一个阈值相似性函数来确定两个响应是否相匹配，并提供了一种根据符合预测进行阈值校准的方法，并在匹配预测的准确性上具有理论保证，这可能具有独立的兴趣。

通过符合性放弃减轻 LLM 幻觉

Mitigating LLM Hallucinations via Conformal Abstention

Retrieval-augmented generation (RAG) grounds large language model (LLM)
output by leveraging external knowledge sources to reduce factual
hallucinations. However, prior works lack a comprehensive evaluation of
different language families, making it challenging to evaluate LLM robustness
against errors in external retrieved knowledge. To overcome this, we establish
NoMIRACL, a human-annotated dataset for evaluating LLM robustness in RAG across
18 typologically diverse languages. NoMIRACL includes both a non-relevant and a
relevant subset. Queries in the non-relevant subset contain passages manually
judged as non-relevant or noisy, whereas queries in the relevant subset include
at least a single judged relevant passage. We measure LLM robustness using two
metrics: (i) hallucination rate, measuring model tendency to hallucinate an
answer, when the answer is not present in passages in the non-relevant subset,
and (ii) error rate, measuring model inaccuracy to recognize relevant passages
in the relevant subset. We build a GPT-4 baseline which achieves a 33.2%
hallucination rate on the non-relevant and a 14.9% error rate on the relevant
subset on average. Our evaluation reveals that GPT-4 hallucinates frequently in
high-resource languages, such as French or English. This work highlights an
important avenue for future research to improve LLM robustness to learn how to
better reject non-relevant information in RAG.

利用外部知识源降低大型语言模型产生的事实幻觉，检索增强生成（RAG）通过对不同语言家族的全面评估，建立 NoMIRACL 数据集来评估 RAG 中 LLM 的鲁棒性。