Despite the prevalence of retrieval-augmented language models (RALMs), the
seamless integration of these models with retrieval mechanisms to enhance
performance in document-based tasks remains challenging. While some
post-retrieval processing Retrieval-Augmented Generation (RAG) methods have
achieved success, most still lack the ability to distinguish pertinent from
extraneous information, leading to potential inconsistencies and reduced
precision in the generated output, which subsequently affects the truthfulness
of the language model's responses. To address these limitations, this work
proposes a novel two-stage consistency learning approach for retrieved
information compression in retrieval-augmented language models to enhance
performance. By incorporating consistency learning, the aim is to generate
summaries that maintain coherence and alignment with the intended semantic
representations of a teacher model while improving faithfulness to the original
retrieved documents. The proposed method is empirically validated across
multiple datasets, demonstrating notable enhancements in precision and
efficiency for question-answering tasks. It outperforms existing baselines and
showcases the synergistic effects of combining contrastive and consistency
learning paradigms within the retrieval-augmented generation framework.

该论文提出了一种新颖的两阶段一致性学习方法，用于改进检索增强语言模型的性能，通过整合一致性学习，目标是生成与教师模型的语义表示保持一致并提高对原始检索文档的忠实程度的摘要，经过多个数据集的经验证明，在问答任务中具有显著的精确性和效率提升，优于现有的基线模型，并展示了在检索增强生成框架中结合对比学习和一致性学习范式的协同效应。