Deep Neural Networks often inherit spurious correlations embedded in training
data and hence may fail to generalize to unseen domains, which have different
distributions from the domain to provide training data. M. Arjovsky et al.
(2019) introduced the concept out-of-distribution (o.o.d.) risk, which is the
maximum risk among all domains, and formulated the issue caused by spurious
correlations as a minimization problem of the o.o.d. risk. Invariant Risk
Minimization (IRM) is considered to be a promising approach to minimize the
o.o.d. risk: IRM estimates a minimum of the o.o.d. risk by solving a bi-level
optimization problem. While IRM has attracted considerable attention with
empirical success, it comes with few theoretical guarantees. Especially, a
solid theoretical guarantee that the bi-level optimization problem gives the
minimum of the o.o.d. risk has not yet been established. Aiming at providing a
theoretical justification for IRM, this paper rigorously proves that a solution
to the bi-level optimization problem minimizes the o.o.d. risk under certain
conditions. The result also provides sufficient conditions on distributions
providing training data and on a dimension of feature space for the bi-leveled
optimization problem to minimize the o.o.d. risk.

深度神经网络存在与训练数据中嵌入的虚假相关性，无法泛化到不同分布的未见域，M. Arjovsky 等人引入了域外风险的概念，并将由虚假相关性引起的问题形式化为域外风险的最小化问题。本文旨在为不变风险最小化提供理论上的验证，证明了双层优化问题的解在一定条件下能使域外风险最小化。同时，文章还提供了训练数据分布和特征空间维度的充分条件，使双层优化问题能最小化域外风险。