Practitioners commonly download pretrained machine learning models from open
repositories and finetune them to fit specific applications. We show that this
practice introduces a new risk of privacy backdoors. By tampering with a
pretrained model's weights, an attacker can fully compromise the privacy of the
finetuning data. We show how to build privacy backdoors for a variety of
models, including transformers, which enable an attacker to reconstruct
individual finetuning samples, with a guaranteed success! We further show that
backdoored models allow for tight privacy attacks on models trained with
differential privacy (DP). The common optimistic practice of training DP models
with loose privacy guarantees is thus insecure if the model is not trusted.
Overall, our work highlights a crucial and overlooked supply chain attack on
machine learning privacy.

预训练机器学习模型存在隐私后门的风险，攻击者能够通过篡改权重完全破坏微调数据的隐私。我们展示了如何为各种模型（包括 transformers）构建隐私后门，进而成功重构个体微调样本。此外，我们还展示了被注入后门的模型能够对使用差分隐私训练的模型进行隐私攻击。因此，如果模型不受信任，使用宽松隐私保证进行差分隐私模型训练的常见乐观实践是不安全的。总的来说，我们的工作突出了对机器学习隐私的一种关键而被忽视的供应链攻击。