Invariant risk minimization (IRM) has recently emerged as a promising
alternative for domain generalization. Nevertheless, the loss function is
difficult to optimize for nonlinear classifiers and the original optimization
objective could fail when pseudo-invariant features and geometric skews exist.
Inspired by IRM, in this paper we propose a novel formulation for domain
generalization, dubbed invariant information bottleneck (IIB). IIB aims at
minimizing invariant risks for nonlinear classifiers and simultaneously
mitigating the impact of pseudo-invariant features and geometric skews.
Specifically, we first present a novel formulation for invariant causal
prediction via mutual information. Then we adopt the variational formulation of
the mutual information to develop a tractable loss function for nonlinear
classifiers. To overcome the failure modes of IRM, we propose to minimize the
mutual information between the inputs and the corresponding representations.
IIB significantly outperforms IRM on synthetic datasets, where the
pseudo-invariant features and geometric skews occur, showing the effectiveness
of proposed formulation in overcoming failure modes of IRM. Furthermore,
experiments on DomainBed show that IIB outperforms $13$ baselines by $0.9\%$ on
average across $7$ real datasets.

本文提出了一种新的不变信息瓶颈（IIB）的域泛化方法，它采用互信息的变分形式来为非线性分类器开发可处理的损失函数，以实现最小化不变风险和减轻伪不变特征和几何偏移对模型的影响。在合成数据集上，IIB 可以显著优于 IRM（不变风险最小化），并且在实际数据集上平均优于 13 个基线方法 0.9％。

不变信息瓶颈用于域泛化

Invariant Information Bottleneck for Domain Generalization

Several recent works have shown that state-of-the-art classifiers are
vulnerable to worst-case (i.e., adversarial) perturbations of the datapoints.
On the other hand, it has been empirically observed that these same classifiers
are relatively robust to random noise. In this paper, we propose to study a
\textit{semi-random} noise regime that generalizes both the random and
worst-case noise regimes. We propose the first quantitative analysis of the
robustness of nonlinear classifiers in this general noise regime. We establish
precise theoretical bounds on the robustness of classifiers in this general
regime, which depend on the curvature of the classifier's decision boundary.
Our bounds confirm and quantify the empirical observations that classifiers
satisfying curvature constraints are robust to random noise. Moreover, we
quantify the robustness of classifiers in terms of the subspace dimension in
the semi-random noise regime, and show that our bounds remarkably interpolate
between the worst-case and random noise regimes. We perform experiments and
show that the derived bounds provide very accurate estimates when applied to
various state-of-the-art deep neural networks and datasets. This result
suggests bounds on the curvature of the classifiers' decision boundaries that
we support experimentally, and more generally offers important insights onto
the geometry of high dimensional classification problems.

本文研究了一种介于随机噪声和最坏情况之间的半随机噪声环境下非线性分类器的鲁棒性，并建立了理论边界，确认并量化了分类器满足曲率约束条件时对随机噪声的鲁棒性。