Large multimodal language models have proven transformative in numerous
applications. However, these models have been shown to memorize and leak
pre-training data, raising serious user privacy and information security
concerns. While data leaks should be prevented, it is also crucial to examine
the trade-off between the privacy protection and model utility of proposed
approaches. In this paper, we introduce PrivQA -- a multimodal benchmark to
assess this privacy/utility trade-off when a model is instructed to protect
specific categories of personal information in a simulated scenario. We also
propose a technique to iteratively self-moderate responses, which significantly
improves privacy. However, through a series of red-teaming experiments, we find
that adversaries can also easily circumvent these protections with simple
jailbreaking methods through textual and/or image inputs. We believe PrivQA has
the potential to support the development of new models with improved privacy
protections, as well as the adversarial robustness of these protections. We
release the entire PrivQA dataset at this https URL

大规模多模式语言模型已在许多应用中证明具有变革性。然而，这些模型已被证明会记忆和泄漏预训练数据，引发用户隐私和信息安全方面的严重关切。我们介绍了 PrivQA - 一个多模式基准，用于评估在模拟场景中指示模型保护特定类别个人信息时，此隐私 / 效用权衡。我们还提出了一种通过迭代自我调节响应的技术，在很大程度上提高了隐私。通过一系列红队实验，我们发现对手可以通过简单的越狱方法绕过这些保护，使用文本和 / 或图像输入。我们相信 PrivQA 有潜力支持开发具有改进隐私保护和对抗鲁棒性的新模型。我们在指定的网址上发布了整个 PrivQA 数据集。