Recent works have introduced GNN-to-MLP knowledge distillation (KD)
frameworks to combine both GNN's superior performance and MLP's fast inference
speed. However, existing KD frameworks are primarily designed for node
classification within single graphs, leaving their applicability to graph
classification largely unexplored. Two main challenges arise when extending KD
for node classification to graph classification: (1) The inherent sparsity of
learning signals due to soft labels being generated at the graph level; (2) The
limited expressiveness of student MLPs, especially in datasets with limited
input feature spaces. To overcome these challenges, we introduce MuGSI, a novel
KD framework that employs Multi-granularity Structural Information for graph
classification. Specifically, we propose multi-granularity distillation loss in
MuGSI to tackle the first challenge. This loss function is composed of three
distinct components: graph-level distillation, subgraph-level distillation, and
node-level distillation. Each component targets a specific granularity of the
graph structure, ensuring a comprehensive transfer of structural knowledge from
the teacher model to the student model. To tackle the second challenge, MuGSI
proposes to incorporate a node feature augmentation component, thereby
enhancing the expressiveness of the student MLPs and making them more capable
learners. We perform extensive experiments across a variety of datasets and
different teacher/student model architectures. The experiment results
demonstrate the effectiveness, efficiency, and robustness of MuGSI. Codes are
publicly available at: \textbf{https://github.com/tianyao-aka/MuGSI.}

构建了一个新的知识蒸馏框架 MuGSI，该框架可以有效地解决图分类中遇到的挑战，并显著提高性能、效率和鲁棒性。

MuGSI：用于图分类的多粒度结构信息压缩 GNN

MuGSI: Distilling GNNs with Multi-Granularity Structural Information for  Graph Classification

Graph Neural Networks (GNNs) have revolutionized graph-based machine
learning, but their heavy computational demands pose challenges for
latency-sensitive edge devices in practical industrial applications. In
response, a new wave of methods, collectively known as GNN-to-MLP Knowledge
Distillation, has emerged. They aim to transfer GNN-learned knowledge to a more
efficient MLP student, which offers faster, resource-efficient inference while
maintaining competitive performance compared to GNNs. However, these methods
face significant challenges in situations with insufficient training data and
incomplete test data, limiting their applicability in real-world applications.
To address these challenges, we propose AdaGMLP, an AdaBoosting GNN-to-MLP
Knowledge Distillation framework. It leverages an ensemble of diverse MLP
students trained on different subsets of labeled nodes, addressing the issue of
insufficient training data. Additionally, it incorporates a Node Alignment
technique for robust predictions on test data with missing or incomplete
features. Our experiments on seven benchmark datasets with different settings
demonstrate that AdaGMLP outperforms existing G2M methods, making it suitable
for a wide range of latency-sensitive real-world applications. We have
submitted our code to the GitHub repository
(this https URL).

AdaGMLP 是一种 AdaBoosting 框架，通过使用多个不同的 MLP 学生对标记节点的不同子集进行训练，解决了训练数据不足的问题，并结合节点对齐技术，提供了对具有缺失或不完整特征的测试数据的鲁棒预测，从而在许多延迟敏感的实际应用中表现优于现有的 G2M 方法。