The current graph neural network (GNN) systems have established a clear trend
of not showing training accuracy results, and directly or indirectly relying on
smaller datasets for evaluations majorly. Our in-depth analysis shows that it
leads to a chain of pitfalls in the system design and evaluation process,
questioning the practicality of many of the proposed system optimizations, and
affecting conclusions and lessons learned. We analyze many single-GPU systems
and show the fundamental impact of these pitfalls. We further develop
hypotheses, recommendations, and evaluation methodologies, and provide future
directions. Finally, a new reference system is developed to establish a new
line of optimizations rooted in solving the system-design pitfalls efficiently
and practically. The proposed design can productively be integrated into prior
works, thereby truly advancing the state-of-the-art.

目前的图神经网络系统在不显示训练准确率结果的同时，直接或间接地依赖较小的数据集进行评估，这导致系统设计和评估过程中存在许多陷阱，质疑了许多已提出的系统优化的实用性及其结论和经验教训，我们对许多单 GPU 系统进行了分析，展示了这些陷阱的基本影响，进一步提出了假设、建议和评估方法，并提供未来发展方向。最后，我们开发了一个新的参考系统，以解决系统设计的陷阱，从而真正推动技术进步。