We study the problem of invariant learning when the environment labels are
unknown. We focus on the invariant representation notion when the Bayes optimal
conditional label distribution is the same across different environments.
Previous work conducts Environment Inference (EI) by maximizing the penalty
term from Invariant Risk Minimization (IRM) framework. The EI step uses a
reference model which focuses on spurious correlations to efficiently reach a
good environment partition. However, it is not clear how to find such a
reference model. In this work, we propose to repeat the EI process and retrain
an ERM model on the \textit{majority} environment inferred by the previous EI
step. Under mild assumptions, we find that this iterative process helps learn a
representation capturing the spurious correlation better than the single step.
This results in better Environment Inference and better Invariant Learning. We
show that this method outperforms baselines on both synthetic and real-world
datasets.

本文研究了在环境标签未知的情况下的不变学习问题。我们提出了一种迭代的环境推断方法，结合环境内部的条件概率分布不变性，提高模型对冗余相关性的理解，从而实现更好的不变学习和环境推断表现。在合成和真实数据集上，实验证明了该方法的优越性。

循环环境推理用于不变性学习

Repeated Environment Inference for Invariant Learning

Factorizing low-rank matrices has many applications in machine learning and
statistics. For probabilistic models in the Bayes optimal setting, a general
expression for the mutual information has been proposed using heuristic
statistical physics computations, and proven in few specific cases. Here, we
show how to rigorously prove the conjectured formula for the symmetric rank-one
case. This allows to express the minimal mean-square-error and to characterize
the detectability phase transitions in a large set of estimation problems
ranging from community detection to sparse PCA. We also show that for a large
set of parameters, an iterative algorithm called approximate message-passing is
Bayes optimal. There exists, however, a gap between what currently known
polynomial algorithms can do and what is expected information theoretically.
Additionally, the proof technique has an interest of its own and exploits three
essential ingredients: the interpolation method introduced in statistical
physics by Guerra, the analysis of the approximate message-passing algorithm
and the theory of spatial coupling and threshold saturation in coding. Our
approach is generic and applicable to other open problems in statistical
estimation where heuristic statistical physics predictions are available.

该论文研究如何采用严谨的方法证明低秩矩阵的对称秩一情况的互信息，进而表达最小均方误差和表征检测相位转变；同时介绍了一种被称为近似消息传递的迭代算法在贝叶斯最优条件下的优越性，并提出了通用的解决其他问题的方法。