In terms of accuracy, Graph Neural Networks (GNNs) are the best architectural
choice for the node classification task. Their drawback in real-world
deployment is the latency that emerges from the neighbourhood processing
operation. One solution to the latency issue is to perform knowledge
distillation from a trained GNN to a Multi-Layer Perceptron (MLP), where the
MLP processes only the features of the node being classified (and possibly some
pre-computed structural information). However, the performance of such MLPs in
both transductive and inductive settings remains inconsistent for existing
knowledge distillation techniques. We propose to address the performance
concerns by using a specially-designed student model instead of an MLP. Our
model, named Routing-by-Memory (RbM), is a form of Mixture-of-Experts (MoE),
with a design that enforces expert specialization. By encouraging each expert
to specialize on a certain region on the hidden representation space, we
demonstrate experimentally that it is possible to derive considerably more
consistent performance across multiple datasets.

通过使用一种特殊设计的学生模型（Routing-by-Memory）来解决节点分类任务中的性能问题，该模型是一种专家混合（Mixture-of-Experts），通过鼓励每个专家在隐藏表示空间的某个特定区域进行专门化，实验证明可以在多个数据集上实现更一致的性能。