Retrieval-augmented generation (RAG) has become a main technique for alleviating hallucinations in large language models (LLMs). Despite the integration of RAG, LLMs may still present unsupported or contradictory claims to the retrieved contents. In order to develop effective hallucination prevention strategies under RAG, it is important to create benchmark datasets that can measure the extent of hallucination. This paper presents RAGTruth, a corpus tailored for analyzing word-level hallucinations in various domains and tasks within the standard RAG frameworks for LLM applications. RAGTruth comprises nearly 18,000 naturally generated responses from diverse LLMs using RAG. These responses have undergone meticulous manual annotations at both the individual cases and word levels, incorporating evaluations of hallucination intensity. We not only benchmark hallucination frequencies across different LLMs, but also critically assess the effectiveness of several existing hallucination detection methodologies. Furthermore, we show that using a high-quality dataset such as RAGTruth, it is possible to finetune a relatively small LLM and achieve a competitive level of performance in hallucination detection when compared to the existing prompt-based approaches using state-of-the-art large language models such as GPT-4.

本文介绍了用于分析不同领域和任务中基于检索增强生成（RAG）大型语言模型（LLMs）中词级幻觉的RAGTruth，这是一个用于测量幻觉程度的基准数据集，并评估了多种现有的幻觉检测方法，在幻觉检测方面，通过使用高质量的数据集如RAGTruth，可以对相对较小的LLM进行微调，并实现与使用GPT-4等最先进的大型语言模型的基于提示的方法相媲美的性能水平。

RAGTruth: 开发可信赖的检索增强语言模型的幻觉语料库