RAG (Retrieval-Augmented Generation) have recently gained significant attention for their enhanced ability to integrate external knowledge sources in open-domain question answering (QA) tasks. However, it remains unclear how these models address fairness concerns, particularly with respect to sensitive attributes such as gender, geographic location, and other demographic factors. First, as language models evolve to prioritize utility, like improving exact match accuracy, fairness may have been largely overlooked. Second, RAG methods are complex pipelines, making it hard to identify and address biases, as each component is optimized for different goals. In this paper, we aim to empirically evaluate fairness in several RAG methods. We propose a fairness evaluation framework tailored to RAG methods, using scenario-based questions and analyzing disparities across demographic attributes. The experimental results indicate that, despite recent advances in utility-driven optimization, fairness issues persist in both the retrieval and generation stages, highlighting the need for more targeted fairness interventions within RAG pipelines. We will release our dataset and code upon acceptance of the paper.

本研究针对检索增强生成(RAG)模型在开放领域问答任务中的公平性缺口进行了探讨，尤其是与性别、地理位置和其他人口因素相关的敏感属性。我们提出了一个专门针对RAG方法的公平性评估框架，通过情境问题分析不同人口属性之间的差异。实验结果表明，尽管在实用性优化上取得了一定进展，但在检索和生成阶段仍存在公平性问题，这强调了在RAG管道中需要更有针对性的公平性干预。

检索增强生成系统中的不公平性评估：RAG是否引入不公平性？