We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et
al. (2019) can fail to capture "natural" invariances, at least when used in its
practical "linear" form, and even on very simple problems which directly follow
the motivating examples for IRM. This can lead to worse generalization on new
environments, even when compared to unconstrained ERM. The issue stems from a
significant gap between the linear variant (as in their concrete method IRMv1)
and the full non-linear IRM formulation. Additionally, even when capturing the
"right" invariances, we show that it is possible for IRM to learn a sub-optimal
predictor, due to the loss function not being invariant across environments.
The issues arise even when measuring invariance on the population
distributions, but are exacerbated by the fact that IRM is extremely fragile to
sampling.

研究表明，Arjovsky 等人的不变风险最小化 (IRM) 公式在其实际的 “线性” 形式下可能无法捕获 “自然” 的不变性，在非常简单的问题上甚至不能解决 IRL 的动机示例，即使与无限制的 ERM 相比，在新环境下可能导致更糟糕的泛化。此外，即使能捕捉 “正确” 的不变性，由于损失函数在不同环境下并不不变，IRM 也可能学习到次优的预测器。这些问题甚至在对总体分布进行不变性度量时出现，并且对抽样极其脆弱。