Machine learning algorithms typically assume that training and test examples
are drawn from the same distribution. However, distribution shift is a common
problem in real-world applications and can cause models to perform dramatically
worse at test time. In this paper, we specifically consider the problems of
subpopulation shifts (e.g., imbalanced data) and domain shifts. While prior
works often seek to explicitly regularize internal representations or
predictors of the model to be domain invariant, we instead aim to learn
invariant predictors without restricting the model's internal representations
or predictors. This leads to a simple mixup-based technique which learns
invariant predictors via selective augmentation called LISA. LISA selectively
interpolates samples either with the same labels but different domains or with
the same domain but different labels. Empirically, we study the effectiveness
of LISA on nine benchmarks ranging from subpopulation shifts to domain shifts,
and we find that LISA consistently outperforms other state-of-the-art methods
and leads to more invariant predictors. We further analyze a linear setting and
theoretically show how LISA leads to a smaller worst-group error.

探讨了在实际应用中常见的基于配分转移的问题，提出了一种选择性增广的简单混合技术 LISA，通过学习不受限制的内部表示或预测器来学习不变的预测器，并在包括亚种人群转移和领域转移的九个基准测试中验证了 LISA 的有效性。

通过选择性增强提高对抗样本的鲁棒性

Improving Out-of-Distribution Robustness via Selective Augmentation

The standard risk minimization paradigm of machine learning is brittle when
operating in environments whose test distributions are different from the
training distribution due to spurious correlations. Training on data from many
environments and finding invariant predictors reduces the effect of spurious
features by concentrating models on features that have a causal relationship
with the outcome. In this work, we pose such invariant risk minimization as
finding the Nash equilibrium of an ensemble game among several environments. By
doing so, we develop a simple training algorithm that uses best response
dynamics and, in our experiments, yields similar or better empirical accuracy
with much lower variance than the challenging bi-level optimization problem of
Arjovsky et al. (2019). One key theoretical contribution is showing that the
set of Nash equilibria for the proposed game are equivalent to the set of
invariant predictors for any finite number of environments, even with nonlinear
classifiers and transformations. As a result, our method also retains the
generalization guarantees to a large set of environments shown in Arjovsky et
al. (2019). The proposed algorithm adds to the collection of successful
game-theoretic machine learning algorithms such as generative adversarial
networks.

该研究提出使用不变风险最小化方法找到多个环境之间的纳什均衡来训练模型，从而降低伪相关性对模型的影响。通过用最佳反应动态来解决这个问题，该方法使得模型在保证泛化的同时提高了训练的稳定性。该算法为博弈论机器学习算法的成功案例之一。