Invariant Causal Prediction (Peters et al., 2016) is a technique for out-of-distribution generalization which assumes that some aspects of the data distribution vary across the training set but that the underlying causal mechanisms remain constant. Recently, Arjovsky et al. (2019) proposed Invariant Risk Minimization (IRM), an objective based on this idea for learning deep, invariant features of data which are a complex function of latent variables; many alternatives have subsequently been suggested. However, formal guarantees for all of these works are severely lacking. In this paper, we present the first analysis of classification under the IRM objective$-$as well as these recently proposed alternatives$-$under a fairly natural and general model. In the linear case, we show simple conditions under which the optimal solution succeeds or, more often, fails to recover the optimal invariant predictor. We furthermore present the very first results in the non-linear regime: we demonstrate that IRM can fail catastrophically unless the test data are sufficiently similar to the training distribution$-$this is precisely the issue that it was intended to solve. Thus, in this setting we find that IRM and its alternatives fundamentally do not improve over standard Empirical Risk Minimization.

在此篇文章中，我们首次针对分类下不变风险最小化 (IRM) 目标以及其最近提出的替代方案，采用自然且广泛的模型进行分析。我们发现在线性情况下，存在简单的条件使最优解成功或更常见的是失败，以至于无法恢复最优的不变预测器。此外，我们还展示了非线性部分的首批结果：除非测试数据与训练分布足够相似，否则 IRM 可能会失败，而这正是它要解决的问题。因此，在这种情况下，我们发现 IRM 及其替代方案从根本上没有改进标准经验风险最小化。

不变风险最小化的风险