Large language models (LLMs) have demonstrated remarkable performance on
various natural language processing tasks. However, they are prone to
generating fluent yet untruthful responses, known as "hallucinations".
Hallucinations can lead to the spread of misinformation and cause harm in
critical applications. Mitigating hallucinations is challenging as they arise
from factors such as noisy data, model overconfidence, lack of knowledge, and
the generation process itself. Recent efforts have attempted to address this
issue through representation editing and decoding algorithms, reducing
hallucinations without major structural changes or retraining. However, these
approaches either implicitly edit LLMs' behavior in latent space or suppress
the tendency to output unfaithful results during decoding instead of explicitly
modeling on hallucination. In this work, we introduce Faithful Finetuning (F2),
a novel method that explicitly models the process of faithful question
answering through carefully designed loss functions during fine-tuning. We
conduct extensive experiments on popular datasets and demonstrate that F2
achieves significant improvements over vanilla models and baselines.

大型语言模型（LLMs）在各种自然语言处理任务上表现出色。然而，它们容易生成流畅但不真实的回复，即 “幻觉”。幻觉可能导致错误信息的传播，并在关键应用中造成伤害。减少幻觉具有挑战性，因为它们来源于嘈杂的数据、模型自信心过度、缺乏知识和生成过程等因素。最近的研究通过表示编辑和解码算法来解决这个问题，实现减少幻觉但不进行重大结构更改或重新训练。然而，这些方法要么隐式地在潜在空间中编辑 LLMs 的行为，要么在解码过程中抑制输出不忠实的结果的倾向，而不是显式地对幻觉进行建模。在本文中，我们介绍了忠实微调（F2），这是一种通过精心设计的损失函数在微调过程中显式地对忠实回答建模的新方法。我们在常用数据集上进行了广泛的实验，并证明 F2 相对于基线模型能够取得显著改进。