Considering the limited internal parametric knowledge, retrieval-augmented
generation (RAG) has been widely used to extend the knowledge scope of large
language models (LLMs). Despite the extensive efforts on RAG research, in
existing methods, LLMs cannot precisely assess the relevance of retrieved
documents, thus likely leading to misleading or even incorrect utilization of
external knowledge (i.e., retrieved documents). To address this issue, in this
paper, we propose REAR, a RElevance-Aware Retrieval-augmented approach for
open-domain question answering (QA). As the key motivation, we aim to enhance
the self-awareness of source relevance for LLMs, so as to adaptively utilize
external knowledge in RAG systems. Specially, we develop a new architecture for
LLM based RAG system, by incorporating a specially designed rank head that
precisely assesses the relevance of retrieved documents. Furthermore, we
propose an improved training method based on bi-granularity relevance fusion
and noise-resistant training. By combining the improvements in both
architecture and training, our proposed REAR can better utilize external
knowledge by effectively perceiving the relevance of retrieved documents.
Experiments on four open-domain QA tasks show that REAR significantly
outperforms previous a number of competitive RAG approaches. Our code and data
can be accessed at this https URL

本文提出了 REAR，这是一种适用于开放领域问答的关联感知检索增强方法，通过在大型语言模型中引入一个能够准确评估检索文档相关性的排名头，以及通过双粒度相关融合和抗干扰训练的改进方法进行训练，REAR 能够更好地利用外部知识并在四个开放领域问答任务中显著优于其他竞争性的 RAG 方法。