Retrieval-augmented generation (RAG) has demonstrated effectiveness in
mitigating the hallucination problem of large language models (LLMs). However,
the difficulty of aligning the retriever with the diverse LLMs' knowledge
preferences inevitably poses an inevitable challenge in developing a reliable
RAG system. To address this issue, we propose DPA-RAG, a universal framework
designed to align diverse knowledge preferences within RAG systems.
Specifically, we initially introduce a preference knowledge construction
pipline and incorporate five novel query augmentation strategies to alleviate
preference data scarcity. Based on preference data, DPA-RAG accomplishes both
external and internal preference alignment: 1) It jointly integrate pair-wise,
point-wise, and contrastive preference alignment abilities into the reranker,
achieving external preference alignment among RAG components. 2) It further
introduces a pre-aligned stage before vanilla Supervised Fine-tuning (SFT),
enabling LLMs to implicitly capture knowledge aligned with their reasoning
preferences, achieving LLMs' internal alignment. Experimental results across
four knowledge-intensive QA datasets demonstrate that DPA-RAG outperforms all
baselines and seamlessly integrates both black-box and open-sourced LLM
readers. Further qualitative analysis and discussions also provide empirical
guidance for achieving reliable RAG systems. Our code is publicly available at
this https URL

DPA-RAG 是一种用于对齐 Retrieval-Augmented Generation 系统中不同知识偏好的通用框架，通过引入查询增强策略和构建偏好知识管道解决了偏好数据稀缺的问题，并在实验证明其优于其他基线系统以及能够无缝集成黑盒和开源的大型语言模型读者。