The goal of domain generalization algorithms is to predict well on distributions different from those seen during training. While a myriad of domain generalization algorithms exist, inconsistencies in experimental conditions -- datasets, architectures, and model selection criteria -- render fair and realistic comparisons difficult. In this paper, we are interested in understanding how useful domain generalization algorithms are in realistic settings. As a first step, we realize that model selection is non-trivial for domain generalization tasks. Contrary to prior work, we argue that domain generalization algorithms without a model selection strategy should be regarded as incomplete. Next, we implement DomainBed, a testbed for domain generalization including seven multi-domain datasets, nine baseline algorithms, and three model selection criteria. We conduct extensive experiments using DomainBed and find that, when carefully implemented, empirical risk minimization shows state-of-the-art performance across all datasets. Looking forward, we hope that the release of DomainBed, along with contributions from fellow researchers, will streamline reproducible and rigorous research in domain generalization.

本文介绍一种在不同训练集下预测性能好的算法——domain generalization algorithms，并提出它们在实际应用中的可用性。作者实现了一个名为 DomainBed 的测试平台，测试了不同数据集、不同基准算法和不同模型选择的效果，并发现在经验风险最小化的情况下，预测性能最优。研究人员们的贡献，加上这个领域的开源，将推动可重复性和严格性研究的发展。

寻找失落的领域泛化