A distribution inference attack aims to infer statistical properties of data
used to train machine learning models. These attacks are sometimes surprisingly
potent, but the factors that impact distribution inference risk are not well
understood and demonstrated attacks often rely on strong and unrealistic
assumptions such as full knowledge of training environments even in supposedly
black-box threat scenarios. To improve understanding of distribution inference
risks, we develop a new black-box attack that even outperforms the best known
white-box attack in most settings. Using this new attack, we evaluate
distribution inference risk while relaxing a variety of assumptions about the
adversary's knowledge under black-box access, like known model architectures
and label-only access. Finally, we evaluate the effectiveness of previously
proposed defenses and introduce new defenses. We find that although noise-based
defenses appear to be ineffective, a simple re-sampling defense can be highly
effective. Code is available at
this https URL

本文研究了分布推理攻击的影响，并开发了一种新的黑盒攻击模型，并评估了不同假设下的推理风险并提出了新的防御措施，发现抗噪防御似乎不起作用，但简单的重新取样防御可以高度有效。