Attribute inference - the process of analyzing publicly available data in
order to uncover hidden information - has become a major threat to privacy,
given the recent technological leap in machine learning. One way to tackle this
threat is to strategically modify one's publicly available data in order to
keep one's private information hidden from attribute inference. We evaluate
people's ability to perform this task, and compare it against algorithms
designed for this purpose. We focus on three attributes: the gender of the
author of a piece of text, the country in which a set of photos was taken, and
the link missing from a social network. For each of these attributes, we find
that people's effectiveness is inferior to that of AI, especially when it comes
to hiding the attribute in question. Moreover, when people are asked to modify
the publicly available information in order to hide these attributes, they are
less likely to make high-impact modifications compared to AI. This suggests
that people are unable to recognize the aspects of the data that are critical
to an inference algorithm. Taken together, our findings highlight the
limitations of relying on human intuition to protect privacy in the age of AI,
and emphasize the need for algorithmic support to protect private information
from attribute inference.

评估了人们修改公开数据以保护隐私的能力，发现人们的效力不如人工智能。强调现代 AI 时代需要算法支持来保护个人信息。

人类直觉作为防御属性推断的手段

Human intuition as a defense against attribute inference

Data modification, either via additional training datasets, data
augmentation, debiasing, and dataset filtering, has been proposed as an
effective solution for generalizing to out-of-domain (OOD) inputs, in both
natural language processing and computer vision literature. However, the effect
of data modification on adversarial robustness remains unclear. In this work,
we conduct a comprehensive study of common data modification strategies and
evaluate not only their in-domain and OOD performance, but also their
adversarial robustness (AR). We also present results on a two-dimensional
synthetic dataset to visualize the effect of each method on the training
distribution. This work serves as an empirical study towards understanding the
relationship between generalizing to unseen domains and defending against
adversarial perturbations. Our findings suggest that more data (either via
additional datasets or data augmentation) benefits both OOD accuracy and AR.
However, data filtering (previously shown to improve OOD accuracy on natural
language inference) hurts OOD accuracy on other tasks such as question
answering and image classification. We provide insights from our experiments to
inform future work in this direction.

本文通过研究数据修正、数据增强、去偏、数据过滤等方法对模型泛化到未知领域和对抗鲁棒性的影响，发现数据的增加（通过增加数据集或数据增强）能够提高模型在未知领域下的准确性和抗干扰能力，而数据过滤则会减少针对特定任务的准确性。

广泛但不健壮？比较数据修改方法对领域外泛化和对抗鲁棒性的影响

Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness

Machine learning systems are deployed in critical settings, but they might
fail in unexpected ways, impacting the accuracy of their predictions. Poisoning
attacks against machine learning induce adversarial modification of data used
by a machine learning algorithm to selectively change its output when it is
deployed. In this work, we introduce a novel data poisoning attack called a
\emph{subpopulation attack}, which is particularly relevant when datasets are
large and diverse. We design a modular framework for subpopulation attacks,
instantiate it with different building blocks, and show that the attacks are
effective for a variety of datasets and machine learning models. We further
optimize the attacks in continuous domains using influence functions and
gradient optimization methods. Compared to existing backdoor poisoning attacks,
subpopulation attacks have the advantage of inducing misclassification in
naturally distributed data points at inference time, making the attacks
extremely stealthy. We also show that our attack strategy can be used to
improve upon existing targeted attacks. We prove that, under some assumptions,
subpopulation attacks are impossible to defend against, and empirically
demonstrate the limitations of existing defenses against our attacks,
highlighting the difficulty of protecting machine learning against this threat.

本文介绍了一种名为 “亚群体攻击” 的数据篡改攻击方法，对于大型和多样化的数据集非常不利，此攻击方法可以使在推理时间内自然分布的数据点产生错误分类，具有缺乏显著性的特点，并表明了现有的保护机制的局限性。