Recent years have witnessed great success in handling graph-related tasks
with Graph Neural Networks (GNNs). Despite their great academic success,
Multi-Layer Perceptrons (MLPs) remain the primary workhorse for practical
industrial applications. One reason for such an academic-industry gap is the
neighborhood-fetching latency incurred by data dependency in GNNs. To reduce
their gaps, Graph Knowledge Distillation (GKD) is proposed, usually based on a
standard teacher-student architecture, to distill knowledge from a large
teacher GNN into a lightweight student GNN or MLP. However, we found in this
paper that neither teachers nor GNNs are necessary for graph knowledge
distillation. We propose a Teacher-Free Graph Self-Distillation (TGS) framework
that does not require any teacher model or GNNs during both training and
inference. More importantly, the proposed TGS framework is purely based on
MLPs, where structural information is only implicitly used to guide dual
knowledge self-distillation between the target node and its neighborhood. As a
result, TGS enjoys the benefits of graph topology awareness in training but is
free from data dependency in inference. Extensive experiments have shown that
the performance of vanilla MLPs can be greatly improved with dual
self-distillation, e.g., TGS improves over vanilla MLPs by 15.54% on average
and outperforms state-of-the-art GKD algorithms on six real-world datasets. In
terms of inference speed, TGS infers 75X-89X faster than existing GNNs and
16X-25X faster than classical inference acceleration methods.

无需师生模型或图神经网络，纯基于多层感知器的图自蒸馏 (TGS) 框架在训练中利用结构信息进行自知识蒸馏，从而在推理中无数据依赖，显著提高了多层感知器的性能，并在六个真实数据集上超过最先进的图知识蒸馏算法。此外，TGS 的推理速度比现有 GNNs 快 75 倍至 89 倍，比传统推理加速方法快 16 倍至 25 倍。