Models trained via empirical risk minimization (ERM) are known to rely on
spurious correlations between labels and task-independent input features,
resulting in poor generalization to distributional shifts. Group
distributionally robust optimization (G-DRO) can alleviate this problem by
minimizing the worst-case loss over a set of pre-defined groups over training
data. G-DRO successfully improves performance of the worst-group, where the
correlation does not hold. However, G-DRO assumes that the spurious
correlations and associated worst groups are known in advance, making it
challenging to apply it to new tasks with potentially multiple unknown spurious
correlations. We propose AGRO -- Adversarial Group discovery for
Distributionally Robust Optimization -- an end-to-end approach that jointly
identifies error-prone groups and improves accuracy on them. AGRO equips G-DRO
with an adversarial slicing model to find a group assignment for training
examples which maximizes worst-case loss over the discovered groups. On the
WILDS benchmark, AGRO results in 8% higher model performance on average on
known worst-groups, compared to prior group discovery approaches used with
G-DRO. AGRO also improves out-of-distribution performance on SST2, QQP, and
MS-COCO -- datasets where potential spurious correlations are as yet
uncharacterized. Human evaluation of ARGO groups shows that they contain
well-defined, yet previously unstudied spurious correlations that lead to model
errors.

通过对抗性组发现的传递稳健优化来提高模型性能，并发现导致模型错误的未研究的虚假相关性。

AGRO: 鲁棒优化中易错群体的对抗性发现

AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization

Social and real-world considerations such as robustness, fairness, social
welfare and multi-agent tradeoffs have given rise to multi-distribution
learning paradigms, such as collaborative, group distributionally robust, and
fair federated learning. In each of these settings, a learner seeks to minimize
its worst-case loss over a set of $n$ predefined distributions, while using as
few samples as possible. In this paper, we establish the optimal sample
complexity of these learning paradigms and give algorithms that meet this
sample complexity. Importantly, our sample complexity bounds exceed that of the
sample complexity of learning a single distribution only by an additive factor
of $n \log(n) / \epsilon^2$. These improve upon the best known sample
complexity of agnostic federated learning by Mohri et al. by a multiplicative
factor of $n$, the sample complexity of collaborative learning by Nguyen and
Zakynthinou by a multiplicative factor $\log n / \epsilon^3$, and give the
first sample complexity bounds for the group DRO objective of Sagawa et al. To
achieve optimal sample complexity, our algorithms learn to sample and learn
from distributions on demand. Our algorithm design and analysis is enabled by
our extensions of stochastic optimization techniques for solving stochastic
zero-sum games. In particular, we contribute variants of Stochastic Mirror
Descent that can trade off between players' access to cheap one-off samples or
more expensive reusable ones.

该论文研究了多分配学习范式的最优样本复杂度，并给出了符合最优样本复杂度的算法。其中，他们的样本复杂度界限超过了仅学习单个分布的学习的样本复杂度。

按需抽样：从多个分布中优化学习

On-Demand Sampling: Learning Optimally from Multiple Distributions

Machine learning systems trained on user-provided data are susceptible to
data poisoning attacks, whereby malicious users inject false training data with
the aim of corrupting the learned model. While recent work has proposed a
number of attacks and defenses, little is understood about the worst-case loss
of a defense in the face of a determined attacker. We address this by
constructing approximate upper bounds on the loss across a broad family of
attacks, for defenders that first perform outlier removal followed by empirical
risk minimization. Our approximation relies on two assumptions: (1) that the
dataset is large enough for statistical concentration between train and test
error to hold, and (2) that outliers within the clean (non-poisoned) data do
not have a strong effect on the model. Our bound comes paired with a candidate
attack that often nearly matches the upper bound, giving us a powerful tool for
quickly assessing defenses on a given dataset. Empirically, we find that even
under a simple defense, the MNIST-1-7 and Dogfish datasets are resilient to
attack, while in contrast the IMDB sentiment dataset can be driven from 12% to
23% test error by adding only 3% poisoned data.

对使用用户提供数据训练的机器学习系统进行的数据毒化攻击问题进行了探讨，构建了数据异常移除后采用经验风险最小化的防御方法下的攻击损失上限并给出了攻击方法，检验了在 MNIST-1-7，Dogfish 数据集上其防御能力较强，而在 IMDB 情感数据集上加入 3% 的毒化数据可将测试误差从 12% 提升至 23%。