Retrieval Augmented Generation (RAG) systems have shown great promise in
natural language processing. However, their reliance on data stored in a
retrieval database, which may contain proprietary or sensitive information,
introduces new privacy concerns. Specifically, an attacker may be able to infer
whether a certain text passage appears in the retrieval database by observing
the outputs of the RAG system, an attack known as a Membership Inference Attack
(MIA). Despite the significance of this threat, MIAs against RAG systems have
yet remained under-explored. This study addresses this gap by introducing an
efficient and easy-to-use method for conducting MIA against RAG systems. We
demonstrate the effectiveness of our attack using two benchmark datasets and
multiple generative models, showing that the membership of a document in the
retrieval database can be efficiently determined through the creation of an
appropriate prompt in both black-box and gray-box settings. Our findings
highlight the importance of implementing security countermeasures in deployed
RAG systems to protect the privacy and security of retrieval databases.

引入了一种高效且易于使用的方法，用于针对检索增强生成（RAG）系统进行成员推断攻击（MIA）；通过使用两个基准数据集和多个生成模型，我们展示了我们攻击的有效性，并且在黑盒和灰盒设置下，可以通过创建适当的提示来高效地确定文档在检索数据库中的成员身份；我们的研究结果突出了实施安全对策以保护检索数据库隐私和安全的重要性。