Out-of-distribution (OOD) learning deals with scenarios in which training and
test data follow different distributions. Although general OOD problems have
been intensively studied in machine learning, graph OOD is only an emerging
area of research. Currently, there lacks a systematic benchmark tailored to
graph OOD method evaluation. In this work, we aim at developing an OOD
benchmark, known as GOOD, for graphs specifically. We explicitly make
distinctions between covariate and concept shifts and design data splits that
accurately reflect different shifts. We consider both graph and node prediction
tasks as there are key differences in designing shifts. Overall, GOOD contains
11 datasets with 17 domain selections. When combined with covariate, concept,
and no shifts, we obtain 51 different splits. We provide performance results on
10 commonly used baseline methods with 10 random runs. This results in 510
dataset-model combinations in total. Our results show significant performance
gaps between in-distribution and OOD settings. Our results also shed light on
different performance trends between covariate and concept shifts by different
methods. Our GOOD benchmark is a growing project and expects to expand in both
quantity and variety of resources as the area develops. The GOOD benchmark can
be accessed via this https URL.

本研究旨在开发用于图形的 OOD 基准（称为 GOOD），以区分协变和概念变化并设计准确反映不同变化的数据拆分。在 10 个常用的基线方法上提供性能结果，结果表明在分布内和 OOD 设置之间存在显着性能差距，本基准将不断发展并扩大资源的数量和种类。