Many applications of machine learning require a model to make accurate
pre-dictions on test examples that are distributionally different from training
ones, while task-specific labels are scarce during training. An effective
approach to this challenge is to pre-train a model on related tasks where data
is abundant, and then fine-tune it on a downstream task of interest. While
pre-training has been effective in many language and vision domains, it remains
an open question how to effectively use pre-training on graph datasets. In this
paper, we develop a new strategy and self-supervised methods for pre-training
Graph Neural Networks (GNNs). The key to the success of our strategy is to
pre-train an expressive GNN at the level of individual nodes as well as entire
graphs so that the GNN can learn useful local and global representations
simultaneously. We systematically study pre-training on multiple graph
classification datasets. We find that naive strategies, which pre-train GNNs at
the level of either entire graphs or individual nodes, give limited improvement
and can even lead to negative transfer on many downstream tasks. In contrast,
our strategy avoids negative transfer and improves generalization significantly
across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC
over non-pre-trained models and achieving state-of-the-art performance for
molecular property prediction and protein function prediction.

本文提出了一种针对图数据集的预训练策略和自监督方法，通过在节点和整个图级别上预训练表达力强的 GNN，以便 GNN 可以同时学习有用的局部和全局表示，并在多个图分类数据集上进行系统研究，发现我们的策略可避免负面转移并显着提高下游任务的泛化性能，最终达到了在分子性质预测和蛋白质功能预测方面的最先进性能。