We study the challenging problem for inference tasks on large-scale graph
datasets of Graph Neural Networks: huge time and memory consumption, and try to
overcome it by reducing reliance on graph structure. Even though distilling
graph knowledge to student MLP is an excellent idea, it faces two major
problems of positional information loss and low generalization. To solve the
problems, we propose a new three-stage multitask distillation framework. In
detail, we use Positional Encoding to capture positional information. Also, we
introduce Neural Heat Kernels responsible for graph data processing in GNN and
utilize hidden layer outputs matching for better performance of student MLP's
hidden layers. To the best of our knowledge, it is the first work to include
hidden layer distillation for student MLP on graphs and to combine graph
Positional Encoding with MLP. We test its performance and robustness with
several settings and draw the conclusion that our work can outperform well with
good stability.

我们研究在大规模图数据集上的图神经网络推理任务中存在的时间和内存消耗的难题，并尝试通过减少对图结构的依赖来克服此问题。为了解决位置信息丢失和低泛化性的两个主要问题，我们提出了一个新的三阶段多任务蒸馏框架，其中使用位置编码来捕捉位置信息，引入神经热核来处理图数据，并利用隐藏层输出进行匹配以提升学生 MLP 的性能。据我们所知，这是第一个在图上为学生 MLP 进行隐藏层蒸馏并将图位置编码与 MLP 相结合的工作。我们通过多种设置测试了其性能和鲁棒性，并得出结论：我们的工作在性能和稳定性方面具有卓越表现。