Semi-supervised node classification in graphs is a fundamental problem in graph mining, and the recently proposed graph neural networks (GNNs) have achieved unparalleled results on this task. Due to their massive success, GNNs have attracted a lot of attention, and many novel architectures have been put forward. In this paper we show that existing evaluation strategies for GNN models have serious shortcomings. We show that using the same train/validation/test splits of the same datasets, as well as making significant changes to the training procedure (e.g. early stopping criteria) precludes a fair comparison of different architectures. We perform a thorough empirical evaluation of four prominent GNN models and show that considering different splits of the data leads to dramatically different rankings of models. Even more importantly, our findings suggest that simpler GNN architectures are able to outperform the more sophisticated ones if the hyperparameters and the training procedure are tuned fairly for all models.

本文探讨了现有的 Graph Neural Networks（GNN）模型评估策略的局限性，通过对四个知名 GNN 模型进行彻底的实证评估，发现只有在公平的条件下比较各种模型的性能才是合理的，简单的 GNN 架构通过超参数和训练程序的公平调整也可以胜过复杂的架构。

图神经网络评估的陷阱