Retrieval-augmented generation (RAG) has become a main technique for
alleviating hallucinations in large language models (LLMs). Despite the
integration of RAG, LLMs may still present unsupported or contradictory claims
to the retrieved contents. In order to develop effective hallucination
prevention strategies under RAG, it is important to create benchmark datasets
that can measure the extent of hallucination. This paper presents RAGTruth, a
corpus tailored for analyzing word-level hallucinations in various domains and
tasks within the standard RAG frameworks for LLM applications. RAGTruth
comprises nearly 18,000 naturally generated responses from diverse LLMs using
RAG. These responses have undergone meticulous manual annotations at both the
individual cases and word levels, incorporating evaluations of hallucination
intensity. We not only benchmark hallucination frequencies across different
LLMs, but also critically assess the effectiveness of several existing
hallucination detection methodologies. Furthermore, we show that using a
high-quality dataset such as RAGTruth, it is possible to finetune a relatively
small LLM and achieve a competitive level of performance in hallucination
detection when compared to the existing prompt-based approaches using
state-of-the-art large language models such as GPT-4.

本文介绍了用于分析不同领域和任务中基于检索增强生成（RAG）大型语言模型（LLMs）中词级幻觉的 RAGTruth，这是一个用于测量幻觉程度的基准数据集，并评估了多种现有的幻觉检测方法，在幻觉检测方面，通过使用高质量的数据集如 RAGTruth，可以对相对较小的 LLM 进行微调，并实现与使用 GPT-4 等最先进的大型语言模型的基于提示的方法相媲美的性能水平。