Despite the rapid advancement of large language models, they remain highly susceptible to generating hallucinations, which significantly hinders their widespread application. Hallucination research requires dynamic and fine-grained evaluation. However, most existing hallucination benchmarks (especially in Chinese language) rely on human annotations, making automatical and cost-effective hallucination evaluation challenging. To address this, we introduce HaluAgent, an agentic framework that automatically constructs fine-grained QA dataset based on some knowledge documents. Our experiments demonstrate that the manually designed rules and prompt optimization can improve the quality of generated data. Using HaluAgent, we construct C-FAITH, a Chinese QA hallucination benchmark created from 1,399 knowledge documents obtained from web scraping, totaling 60,702 entries. We comprehensively evaluate 16 mainstream LLMs with our proposed C-FAITH, providing detailed experimental results and analysis.

本研究解决了大型语言模型在生成幻觉时的评价不足，尤其是在中文领域。通过引入HaluAgent框架，自动构建了细粒度问答数据集C-FAITH，使得幻觉评估更加高效和经济。实验结果表明，该基准能够有效评估主流大型语言模型的性能，推动了相关领域的研究进展。

C-FAITH：自动化幻觉评估的中文细粒度基准