Given data from diverse sets of distinct distributions, domain generalization
aims to learn models that generalize to unseen distributions. A common approach
is designing a data-driven surrogate penalty to capture generalization and
minimize the empirical risk jointly with the penalty. We argue that a
significant failure mode of this recipe is an excess risk due to an erroneous
penalty or hardness in joint optimization. We present an approach that
eliminates this problem. Instead of jointly minimizing empirical risk with the
penalty, we minimize the penalty under the constraint of optimality of the
empirical risk. This change guarantees that the domain generalization penalty
cannot impair optimization of the empirical risk, i.e., in-distribution
performance. To solve the proposed optimization problem, we demonstrate an
exciting connection to rate-distortion theory and utilize its tools to design
an efficient method. Our approach can be applied to any penalty-based domain
generalization method, and we demonstrate its effectiveness by applying it to
three examplar methods from the literature, showing significant improvements.

通过最小化约束条件下的惩罚，而不是与经验风险同时最小化，我们提出了一种解决域泛化领域中过度风险的方法，并通过将我们的方法应用于文献中的三种方法来证明其有效性。