Modern machine learning (ML) ecosystems offer a surging number of ML frameworks and code repositories that can greatly facilitate the development of ML models. Today, even ordinary data holders who are not ML experts can apply off-the-shelf codebase to build high-performance ML models on their data, many of which are sensitive in nature (e.g., clinical records). In this work, we consider a malicious ML provider who supplies model-training code to the data holders, does not have access to the training process, and has only black-box query access to the resulting model. In this setting, we demonstrate a new form of membership inference attack that is strictly more powerful than prior art. Our attack empowers the adversary to reliably de-identify all the training samples (average >99% attack TPR@0.1% FPR), and the compromised models still maintain competitive performance as their uncorrupted counterparts (average <1% accuracy drop). Moreover, we show that the poisoned models can effectively disguise the amplified membership leakage under common membership privacy auditing, which can only be revealed by a set of secret samples known by the adversary. Overall, our study not only points to the worst-case membership privacy leakage, but also unveils a common pitfall underlying existing privacy auditing methods, which calls for future efforts to rethink the current practice of auditing membership privacy in machine learning models.

现代机器学习（ML）生态系统提供了大量的ML框架和代码库，可以极大地促进ML模型的开发。本研究考虑了恶意ML提供者供应模型训练代码给数据持有者的情况，该提供者无法访问训练过程，只能以黑盒查询方式访问结果模型。我们展示了一种新形式的成员推断攻击，比以往的攻击更强大，使对手能够可靠地取消识别所有训练样本，并且被攻击的模型仍然保持与未受损对照模型相当的性能。此外，我们还展示了被污染的模型可以在常见的成员隐私审核下有效伪装被放大的成员泄漏，只有对手知道的一组秘密样本才能揭示。总体而言，我们的研究不仅指出了最坏情况下的成员隐私泄漏，还揭示了现有隐私审核方法的一个常见问题，需要未来努力重新思考机器学习模型中的隐私审核实践。

深度学习模型中实施成员推断攻击的方法