Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios. However, their emergence also raises significant data security concerns, given the potential inclusion of sensitive information, such as private photos and medical records, in their training datasets. Detecting inappropriately used data in VLLMs remains a critical and unresolved issue, mainly due to the lack of standardized datasets and suitable methodologies. In this study, we introduce the first membership inference attack (MIA) benchmark tailored for various VLLMs to facilitate training data detection. Then, we propose a novel MIA pipeline specifically designed for token-level image detection. Lastly, we present a new metric called MaxR\'enyi-K%, which is based on the confidence of the model output and applies to both text and image data. We believe that our work can deepen the understanding and methodology of MIAs in the context of VLLMs. Our code and datasets are available at https://github.com/LIONS-EPFL/VL-MIA.

本研究解决了大规模视觉-语言模型（VLLMs）在敏感数据使用方面的安全隐患。我们提出了首个针对VLLMs的成员推断攻击（MIA）基准，开发了专门的MIA管道，并引入了一种新指标MaxR\'enyi-K%，从而有助于增强对VLLMs的数据检测能力。这项工作有望改善对成员推断攻击的理解及其在VLLMs中的应用方法。

针对大规模视觉-语言模型的成员推断攻击