Deep learning has achieved tremendous success with independent and
identically distributed (i.i.d.) data. However, the performance of neural
networks often degenerates drastically when encountering out-of-distribution
(OoD) data, i.e., when training and test data are sampled from different
distributions. While a plethora of algorithms have been proposed for OoD
generalization, our understanding of the data used to train and evaluate these
algorithms remains stagnant. In this work, we first identify and measure two
distinct kinds of distribution shifts that are ubiquitous in various datasets.
Next, through extensive experiments, we compare OoD generalization algorithms
across two groups of benchmarks, each dominated by one of the distribution
shifts, revealing their strengths on one shift as well as limitations on the
other shift. Overall, we position existing datasets and algorithms from
different research areas seemingly unconnected into the same coherent picture.
It may serve as a foothold that can be resorted to by future OoD generalization
research. Our code is available at this https URL

通过对分布偏移数据的分类和对其相关算法的大量实验比较，对已有数据集和算法进行了分类整合，为未来的迁移学习和数据偏移研究提供了参考。