Incorporating factual knowledge in knowledge graph is regarded as a promising
approach for mitigating the hallucination of large language models (LLMs).
Existing methods usually only use the user's input to query the knowledge
graph, thus failing to address the factual hallucination generated by LLMs
during its reasoning process. To address this problem, this paper proposes
Knowledge Graph-based Retrofitting (KGR), a new framework that incorporates
LLMs with KGs to mitigate factual hallucination during the reasoning process by
retrofitting the initial draft responses of LLMs based on the factual knowledge
stored in KGs. Specifically, KGR leverages LLMs to extract, select, validate,
and retrofit factual statements within the model-generated responses, which
enables an autonomous knowledge verifying and refining procedure without any
additional manual efforts. Experiments show that KGR can significantly improve
the performance of LLMs on factual QA benchmarks especially when involving
complex reasoning processes, which demonstrates the necessity and effectiveness
of KGR in mitigating hallucination and enhancing the reliability of LLMs.

知识图谱的积极运用可以有效减轻大型语言模型的虚构幻觉，本文提出了一种基于知识图谱的改进框架，通过将知识图谱与语言模型相结合，修改模型生成的初步回答，从而验证和完善其中的事实陈述，实验证明该框架在解决复杂推理过程中显著提高了语言模型在事实问答基准测试中的性能，有效减轻了虚构幻觉，增强了语言模型的可信度。

基于自治知识图谱改进的减轻大型语言模型幻觉

Mitigating Large Language Model Hallucinations via Autonomous Knowledge  Graph-based Retrofitting

Retrieval-augmented language models (RALMs) represent a substantial
advancement in the capabilities of large language models, notably in reducing
factual hallucination by leveraging external knowledge sources. However, the
reliability of the retrieved information is not always guaranteed. The
retrieval of irrelevant data can lead to misguided responses, and potentially
causing the model to overlook its inherent knowledge, even when it possesses
adequate information to address the query. Moreover, standard RALMs often
struggle to assess whether they possess adequate knowledge, both intrinsic and
retrieved, to provide an accurate answer. In situations where knowledge is
lacking, these systems should ideally respond with "unknown" when the answer is
unattainable. In response to these challenges, we introduces Chain-of-Noting
(CoN), a novel approach aimed at improving the robustness of RALMs in facing
noisy, irrelevant documents and in handling unknown scenarios. The core idea of
CoN is to generate sequential reading notes for retrieved documents, enabling a
thorough evaluation of their relevance to the given question and integrating
this information to formulate the final answer. We employed ChatGPT to create
training data for CoN, which was subsequently trained on an LLaMa-2 7B model.
Our experiments across four open-domain QA benchmarks show that RALMs equipped
with CoN significantly outperform standard RALMs. Notably, CoN achieves an
average improvement of +7.9 in EM score given entirely noisy retrieved
documents and +10.5 in rejection rates for real-time questions that fall
outside the pre-training knowledge scope.

Retrieval-augmented language models (RALMs) with Chain-of-Noting (CoN) significantly improve the robustness in handling noisy, irrelevant documents and unknown scenarios, outperforming standard RALMs on open-domain QA benchmarks.