Adversarial examples are malicious inputs crafted to cause a model to
misclassify them. Their most common instantiation, "perturbation-based"
adversarial examples introduce changes to the input that leave its true label
unchanged, yet result in a different model prediction. Conversely,
"invariance-based" adversarial examples insert changes to the input that leave
the model's prediction unaffected despite the underlying input's label having
changed.
In this paper, we demonstrate that robustness to perturbation-based
adversarial examples is not only insufficient for general robustness, but
worse, it can also increase vulnerability of the model to invariance-based
adversarial examples. In addition to analytical constructions, we empirically
study vision classifiers with state-of-the-art robustness to perturbation-based
adversaries constrained by an $\ell_p$ norm. We mount attacks that exploit
excessive model invariance in directions relevant to the task, which are able
to find adversarial examples within the $\ell_p$ ball. In fact, we find that
classifiers trained to be $\ell_p$-norm robust are more vulnerable to
invariance-based adversarial examples than their undefended counterparts.
Excessive invariance is not limited to models trained to be robust to
perturbation-based $\ell_p$-norm adversaries. In fact, we argue that the term
adversarial example is used to capture a series of model limitations, some of
which may not have been discovered yet. Accordingly, we call for a set of
precise definitions that taxonomize and address each of these shortcomings in
learning.

本文演示了对扰动型对抗样本的稳健性不仅不足以实现普遍的稳健性，而且它还会增加模型对于不变性型对抗样本的脆弱性，并呼吁一组精确的定义来对学习中的这些限制进行分类和解决。

利用范数界限对抗鲁棒性引起的过度不变性

Exploiting Excessive Invariance caused by Norm-Bounded Adversarial  Robustness

Membership inference attacks seek to infer membership of individual training
instances of a model to which an adversary has black-box access through a
machine learning-as-a-service API. In providing an in-depth characterization of
membership privacy risks against machine learning models, this paper presents a
comprehensive study towards demystifying membership inference attacks from two
complimentary perspectives. First, we provide a generalized formulation of the
development of a black-box membership inference attack model. Second, we
characterize the importance of model choice on model vulnerability through a
systematic evaluation of a variety of machine learning models and model
combinations using multiple datasets. Through formal analysis and empirical
evidence from extensive experimentation, we characterize under what conditions
a model may be vulnerable to such black-box membership inference attacks. We
show that membership inference vulnerability is data-driven and corresponding
attack models are largely transferable. Though different model types display
different vulnerabilities to membership inference, so do different datasets.
Our empirical results additionally show that (1) using the type of target model
under attack within the attack model may not increase attack effectiveness and
(2) collaborative learning exposes vulnerabilities to membership inference
risks when the adversary is a participant. We also discuss countermeasure and
mitigation strategies.

该论文通过对机器学习模型逐一评估，探究其在会员隐私方面存在的风险。研究表明，攻击模型的效果主要由数据驱动，受数据集的影响较大。在攻击过程中，抗攻击模型的选择和参与者的数量也是影响因素之一。最后，论文给出了相应的对策和缓解策略。