Recent trends in lower precision, e.g. half-precision floating point, training have shown improved system performance and reduced memory usage for Deep Learning while maintaining accuracy. However, current GNN systems cannot achieve such goals for GNN, as our analyses show that they massively underperform while showing abnormal accuracy when using half-precision. These systems suffer from value overflow issues due to lowered precision, under-utilization of hardware resources, and poor training performance. To mitigate this, we introduce HalfGNN, a half-precision based GNN system. HalfGNN proposes novel techniques: new vector operations for half-precision data types that improve data load and reduction performance, and discretized SpMM that overcomes the value overflow and natively provides workload balancing. Such techniques improve hardware utilization, reduce memory usage, and remove atomic writes. Evaluations show that HalfGNN achieves on average of 2.30X speedup in training time over DGL (float-based) for GAT, GCN, and GIN respectively while achieving similar accuracy, and saving 2.67X memory.

本研究针对现有图神经网络(GNN)系统在使用半精度浮点训练时表现不佳的问题，提出了一种新的解决方案。通过开发HalfGNN系统，采用新型向量操作和离散化的稀疏矩阵乘法(SpMM)技术，该方法显著提升了硬件利用率，减少了内存使用，并提高了训练效率，最终实现比传统方法提高2.30倍的训练速度，同时保持类似的准确性。

使用半精度进行图神经网络训练