Recent studies propose membership inference (MI) attacks on deep models. Despite the moderate accuracy of such MI attacks, we show that the way the attack accuracy is reported is often misleading and a simple blind attack which is highly unreliable and inefficient in reality can often represent similar accuracy. We show that the current MI attack models can only identify the membership of misclassified samples with mediocre accuracy at best, which only constitute a very small portion of training samples. We analyze several new features that have not been explored for membership inference before, including distance to the decision boundary and gradient norms, and conclude that deep models' responses are mostly indistinguishable among train and non-train samples. Moreover, in contrast with general intuition that deeper models have a capacity to memorize training samples, and, hence, they are more vulnerable to membership inference, we find no evidence to support that and in some cases deeper models are often harder to launch membership inference attack on. Furthermore, despite the common belief, we show that overfitting does not necessarily lead to higher degree of membership leakage. We conduct experiments on MNIST, CIFAR-10, CIFAR-100, and ImageNet, using various model architecture, including LeNet, ResNet, DenseNet, InceptionV3, and Xception. Source code: https://github.com/shrezaei/MI-Attack}{\color{blue} {https://github.com/shrezaei/MI-Attack}.

该论文研究深度学习中的成员推理攻击，指出现有攻击方法的报告中存在高误报率(FAR)和针对误分样本的较弱识别能力，并探索了距离决策边界和梯度范数等新特征，证明其对误分类样本分类能力普遍一般，最后通过多项实验证明了现有的MI攻击较难同时实现高准确率和低FAR。

隶属推断攻击的难度