It is commonplace to produce application-specific models by fine-tuning large pre-trained models using a small bespoke dataset. The widespread availability of foundation model checkpoints on the web poses considerable risks, including the vulnerability to backdoor attacks. In this paper, we unveil a new vulnerability: the privacy backdoor attack. This black-box privacy attack aims to amplify the privacy leakage that arises when fine-tuning a model: when a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. We conduct extensive experiments on various datasets and models, including both vision-language models (CLIP) and large language models, demonstrating the broad applicability and effectiveness of such an attack. Additionally, we carry out multiple ablation studies with different fine-tuning methods and inference strategies to thoroughly analyze this new threat. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.

利用小型专用数据集微调大型预训练模型来生成特定应用模型是常见的做法。然而，我们揭示了一种新的漏洞：隐私后门攻击，通过该攻击，在微调受后门影响的模型时，训练数据的隐私泄露率会显著增加。我们在不同数据集和模型上进行了大量实验证明了这种攻击的广泛适用性和有效性，并通过不同微调方法和推断策略进行了多次消融研究以全面分析这个新威胁。我们的发现突出了机器学习社区的重要隐私问题，并呼吁重新评估使用开源预训练模型的安全协议。

隐私后门：通过污染预训练模型增强成员推断