We develop a principled procedure for determining when a large language model
(LLM) should abstain from responding (e.g., by saying "I don't know") in a
general domain, instead of resorting to possibly "hallucinating" a non-sensical
or incorrect answer. Building on earlier approaches that use self-consistency
as a more reliable measure of model confidence, we propose using the LLM itself
to self-evaluate the similarity between each of its sampled responses for a
given query. We then further leverage conformal prediction techniques to
develop an abstention procedure that benefits from rigorous theoretical
guarantees on the hallucination rate (error rate). Experimentally, our
resulting conformal abstention method reliably bounds the hallucination rate on
various closed-book, open-domain generative question answering datasets, while
also maintaining a significantly less conservative abstention rate on a dataset
with long responses (Temporal Sequences) compared to baselines using
log-probability scores to quantify uncertainty, while achieveing comparable
performance on a dataset with short answers (TriviaQA). To evaluate the
experiments automatically, one needs to determine if two responses are
equivalent given a question. Following standard practice, we use a thresholded
similarity function to determine if two responses match, but also provide a
method for calibrating the threshold based on conformal prediction, with
theoretical guarantees on the accuracy of the match prediction, which might be
of independent interest.

我们开发了一个原则性的方法，确定一个大型语言模型在一般领域中，何时应该放弃回答而选择回答 “我不知道”，而不是胡言乱语或错误答案。通过使用自洽性作为一种更可靠的模型置信度度量的早期方法的基础上，我们提出使用语言模型本身来自我评估其对于给定查询的每个采样响应之间的相似性。然后，我们进一步利用符合预测技术来开发一种放弃回答的方法，该方法在误报率（错误率）上具有严格的理论保证。在实验中，我们的符合预测放弃方法可可靠地限制了各种闭书、开放领域的生成问答数据集的胡言乱语率，同时与基于对数概率分数量化不确定性的基线相比，在长答案的数据集（时间序列）上保持了一个显著较低的放弃率，同时在短答案的数据集（TriviaQA）上实现了可比较的性能。为了自动评估实验，需要确定两个答案在给定问题下是否等价。遵循标准做法，我们使用一个阈值相似性函数来确定两个响应是否相匹配，并提供了一种根据符合预测进行阈值校准的方法，并在匹配预测的准确性上具有理论保证，这可能具有独立的兴趣。