While Large Language Models (LLMs) have proven to be exceptional on a variety
of tasks after alignment, they may still produce responses that contradict the
context or world knowledge confidently, a phenomenon known as
``hallucination''. In this paper, we demonstrate that reducing the
inconsistency between the external knowledge encapsulated in the training data
and the intrinsic knowledge inherited in the pretraining corpus could mitigate
hallucination in alignment. Specifically, we introduce a novel knowledge
consistent alignment (KCA) approach, which involves automatically formulating
examinations based on external knowledge for accessing the comprehension of
LLMs. For data encompassing knowledge inconsistency, KCA implements several
simple yet efficient strategies for processing. We illustrate the superior
performance of the proposed KCA approach in mitigating hallucinations across
six benchmarks using LLMs of different backbones and scales. Furthermore, we
confirm the correlation between knowledge inconsistency and hallucination,
signifying the effectiveness of reducing knowledge inconsistency in alleviating
hallucinations. Our code, model weights, and data are public at
https://github.com/fanqiwan/KCA.

通过知识一致性对齐的新方法，减少外部知识与预训练语料库中的内在知识之间的不一致，以减轻大型语言模型在对齐过程中出现的幻觉现象，并证实了知识不一致性与幻觉之间的相关性。