Graph neural networks (GNNs) have been demonstrated to be powerful in
modeling graph-structured data. However, training GNNs usually requires
abundant task-specific labeled data, which is often arduously expensive to
obtain. One effective way to reduce the labeling effort is to pre-train an
expressive GNN model on unlabeled data with self-supervision and then transfer
the learned model to downstream tasks with only a few labels. In this paper, we
present the GPT-GNN framework to initialize GNNs by generative pre-training.
GPT-GNN introduces a self-supervised attributed graph generation task to
pre-train a GNN so that it can capture the structural and semantic properties
of the graph. We factorize the likelihood of the graph generation into two
components: 1) Attribute Generation and 2) Edge Generation. By modeling both
components, GPT-GNN captures the inherent dependency between node attributes
and graph structure during the generative process. Comprehensive experiments on
the billion-scale Open Academic Graph and Amazon recommendation data
demonstrate that GPT-GNN significantly outperforms state-of-the-art GNN models
without pre-training by up to 9.1% across various downstream tasks.

本文介绍了一种名为 GPT-GNN 的框架，用于解决 Graph Neural Networks 训练问题，通过预训练一个能够自监督地学习到结构和语义属性的 GNN 模型，然后将其传输到 Downstream Tasks 中，从而显著提高了性能。