Recently, the paradigm of pre-training and fine-tuning graph neural networks
has been intensively studied and applied in a wide range of graph mining tasks.
Its success is generally attributed to the structural consistency between
pre-training and downstream datasets, which, however, does not hold in many
real-world scenarios. Existing works have shown that the structural divergence
between pre-training and downstream graphs significantly limits the
transferability when using the vanilla fine-tuning strategy. This divergence
leads to model overfitting on pre-training graphs and causes difficulties in
capturing the structural properties of the downstream graphs. In this paper, we
identify the fundamental cause of structural divergence as the discrepancy of
generative patterns between the pre-training and downstream graphs.
Furthermore, we propose G-Tuning to preserve the generative patterns of
downstream graphs. Given a downstream graph G, the core idea is to tune the
pre-trained GNN so that it can reconstruct the generative patterns of G, the
graphon W. However, the exact reconstruction of a graphon is known to be
computationally expensive. To overcome this challenge, we provide a theoretical
analysis that establishes the existence of a set of alternative graphons called
graphon bases for any given graphon. By utilizing a linear combination of these
graphon bases, we can efficiently approximate W. This theoretical finding forms
the basis of our proposed model, as it enables effective learning of the
graphon bases and their associated coefficients. Compared with existing
algorithms, G-Tuning demonstrates an average improvement of 0.5% and 2.6% on
in-domain and out-of-domain transfer learning experiments, respectively.

本研究旨在解决预训练图神经网络和下游图数据之间结构差异的问题，提出了 G-Tuning 方法，通过调整预训练 GNN 来保留下游图的生成模式，有效改善了领域内和领域外的迁移学习实验结果。