Invariant risk minimization (IRM) has recently emerged as a promising
alternative for domain generalization. Nevertheless, the loss function is
difficult to optimize for nonlinear classifiers and the original optimization
objective could fail when pseudo-invariant features and geometric skews exist.
Inspired by IRM, in this paper we propose a novel formulation for domain
generalization, dubbed invariant information bottleneck (IIB). IIB aims at
minimizing invariant risks for nonlinear classifiers and simultaneously
mitigating the impact of pseudo-invariant features and geometric skews.
Specifically, we first present a novel formulation for invariant causal
prediction via mutual information. Then we adopt the variational formulation of
the mutual information to develop a tractable loss function for nonlinear
classifiers. To overcome the failure modes of IRM, we propose to minimize the
mutual information between the inputs and the corresponding representations.
IIB significantly outperforms IRM on synthetic datasets, where the
pseudo-invariant features and geometric skews occur, showing the effectiveness
of proposed formulation in overcoming failure modes of IRM. Furthermore,
experiments on DomainBed show that IIB outperforms $13$ baselines by $0.9\%$ on
average across $7$ real datasets.

本文提出了一种新的不变信息瓶颈（IIB）的域泛化方法，它采用互信息的变分形式来为非线性分类器开发可处理的损失函数，以实现最小化不变风险和减轻伪不变特征和几何偏移对模型的影响。在合成数据集上，IIB 可以显著优于 IRM（不变风险最小化），并且在实际数据集上平均优于 13 个基线方法 0.9％。