Retrieval-augmented language models (RALMs) have shown strong performance and wide applicability in knowledge-intensive tasks. However, there are significant trustworthiness concerns as RALMs are prone to generating unfaithful outputs, including baseless information or contradictions with the retrieved context. This paper proposes SynCheck, a lightweight monitor that leverages fine-grained decoding dynamics including sequence likelihood, uncertainty quantification, context influence, and semantic alignment to synchronously detect unfaithful sentences. By integrating efficiently measurable and complementary signals, SynCheck enables accurate and immediate feedback and intervention, achieving 0.85 AUROC in detecting faithfulness errors across six long-form retrieval-augmented generation tasks, improving prior best method by 4%. Leveraging SynCheck, we further introduce FOD, a faithfulness-oriented decoding algorithm guided by beam search for long-form retrieval-augmented generation. Empirical results demonstrate that FOD outperforms traditional strategies such as abstention, reranking, or contrastive decoding significantly in terms of faithfulness, achieving over 10% improvement across six datasets.

本文提出了SynCheck，一种轻量级监控器，可以实时检测不忠实的句子，并通过集成可衡量和互补的信号提供准确和即时的反馈和干预，以改善在长篇检索增强生成任务中对不忠实性错误的检测。此外，作者还介绍了基于SynCheck的FOD解码算法，通过引导束搜索，在长篇检索增强生成任务中实现了显著的忠实性改进。

用于可信检索扩增生成的同步忠实监控