Evaluating the performance of a well-trained GNN model on real-world graphs is a pivotal step for reliable GNN online deployment and serving. Due to a lack of test node labels and unknown potential training-test graph data distribution shifts, conventional model evaluation encounters limitations in calculating performance metrics (e.g., test error) and measuring graph data-level discrepancies, particularly when the training graph used for developing GNNs remains unobserved during test time. In this paper, we study a new research problem, online GNN evaluation, which aims to provide valuable insights into the well-trained GNNs's ability to effectively generalize to real-world unlabeled graphs under the test-time graph distribution shifts. Concretely, we develop an effective learning behavior discrepancy score, dubbed LeBeD, to estimate the test-time generalization errors of well-trained GNN models. Through a novel GNN re-training strategy with a parameter-free optimality criterion, the proposed LeBeD comprehensively integrates learning behavior discrepancies from both node prediction and structure reconstruction perspectives. This enables the effective evaluation of the well-trained GNNs' ability to capture test node semantics and structural representations, making it an expressive metric for estimating the generalization error in online GNN evaluation. Extensive experiments on real-world test graphs under diverse graph distribution shifts could verify the effectiveness of the proposed method, revealing its strong correlation with ground-truth test errors on various well-trained GNN models.

评估训练良好的GNN模型在现实世界的图上的性能是可靠的GNN在线部署和服务的关键一步。本论文研究了在线GNN评估这一新的研究问题，旨在为良好训练的GNN模型在测试时能否有效地推广到实际未标记图中提供有价值的见解。通过开发一种有效的学习行为差异度分数LeBeD，该分数从节点预测和结构重构的角度全面集成了学习行为差异度，使得可以有效评估训练良好的GNN模型捕捉测试节点语义和结构表示的能力，从而成为在线GNN评估中估计泛化误差的一个有表现力的指标。大量针对不同图分布转移的实际测试图进行的实验证明了所提方法的有效性，并揭示了它与各种训练良好的GNN模型的地面真实测试误差之间的强相关性。

测试时间图分布偏移下的在线GNN评估