Retrieval Augmented Generation (RAG) systems have shown great promise in natural language processing. However, their reliance on data stored in a retrieval database, which may contain proprietary or sensitive information, introduces new privacy concerns. Specifically, an attacker may be able to infer whether a certain text passage appears in the retrieval database by observing the outputs of the RAG system, an attack known as a Membership Inference Attack (MIA). Despite the significance of this threat, MIAs against RAG systems have yet remained under-explored. This study addresses this gap by introducing an efficient and easy-to-use method for conducting MIA against RAG systems. We demonstrate the effectiveness of our attack using two benchmark datasets and multiple generative models, showing that the membership of a document in the retrieval database can be efficiently determined through the creation of an appropriate prompt in both black-box and gray-box settings. Our findings highlight the importance of implementing security countermeasures in deployed RAG systems to protect the privacy and security of retrieval databases.

引入了一种高效且易于使用的方法，用于针对检索增强生成（RAG）系统进行成员推断攻击（MIA）；通过使用两个基准数据集和多个生成模型，我们展示了我们攻击的有效性，并且在黑盒和灰盒设置下，可以通过创建适当的提示来高效地确定文档在检索数据库中的成员身份；我们的研究结果突出了实施安全对策以保护检索数据库隐私和安全的重要性。

检索增强生成中的成员推断攻击:我的数据是否在您的检索数据库中?