Despite impressive success in many tasks, deep learning models are shown to rely on spurious features, which will catastrophically fail when generalized to out-of-distribution (OOD) data. Invariant Risk Minimization (IRM) is proposed to alleviate this issue by extracting domain-invariant features for OOD generalization. Nevertheless, recent work shows that IRM is only effective for a certain type of distribution shift (e.g., correlation shift) while it fails for other cases (e.g., diversity shift). Meanwhile, another thread of method, Adversarial Training (AT), has shown better domain transfer performance, suggesting that it has the potential to be an effective candidate for extracting domain-invariant features. This paper investigates this possibility by exploring the similarity between the IRM and AT objectives. Inspired by this connection, we propose Domainwise Adversarial Training (DAT), an AT-inspired method for alleviating distribution shift by domain-specific perturbations. Extensive experiments show that our proposed DAT can effectively remove domain-varying features and improve OOD generalization under both correlation shift and diversity shift.

本文探究一种基于对抗训练的深度学习模型方法，称为Domainwise Adversarial Training (DAT)，来解决深度学习模型在广义分布中失败的问题，该方法借鉴了Invariant Risk Minimization (IRM) 和Adversarial Training (AT)方法，并通过实验证明，我们提出的DAT方法在解决不同广义分布转移方案上表现均可。

关于不变性学习和对抗训练与超出分布泛化的联系