Large Language Models (LLMs) have demonstrated their ability to collaborate
effectively with humans in real-world scenarios. However, LLMs are apt to
generate hallucinations, i.e., makeup incorrect text and unverified
information, which can cause significant damage when deployed for
mission-critical tasks. In this paper, we propose a self-check approach based
on reverse validation to detect factual errors automatically in a zero-resource
fashion. To facilitate future studies and assess different methods, we
construct a hallucination detection benchmark, which is generated by ChatGPT
and annotated by human annotators. Contrasting previous studies of
zero-resource hallucination detection, our method and benchmark concentrate on
passage-level detection instead of sentence-level. We empirically evaluate our
method and existing zero-resource detection methods on different domains of
benchmark to explore the implicit relation between hallucination and training
data. Furthermore, we manually analyze some hallucination cases that LLM failed
to capture, revealing the shared limitation of zero-resource methods.

本研究提出了一种基于反向验证的自检方法，以零资源方式自动检测事实错误，并构建了一个基于 ChatGPT 生成的、由人工注释的幻觉检测基准，在段落级别进行研究和评估不同方法，揭示了零资源方法的共同局限。