Full-batch training on Graph Neural Networks (GNN) to learn the structure of
large graphs is a critical problem that needs to scale to hundreds of compute
nodes to be feasible. It is challenging due to large memory capacity and
bandwidth requirements on a single compute node and high communication volumes
across multiple nodes. In this paper, we present DistGNN that optimizes the
well-known Deep Graph Library (DGL) for full-batch training on CPU clusters via
an efficient shared memory implementation, communication reduction using a
minimum vertex-cut graph partitioning algorithm and communication avoidance
using a family of delayed-update algorithms. Our results on four common GNN
benchmark datasets: Reddit, OGB-Products, OGB-Papers and Proteins, show up to
3.7x speed-up using a single CPU socket and up to 97x speed-up using 128 CPU
sockets, respectively, over baseline DGL implementations running on a single
CPU socket

该论文介绍 DistGNN，它使用共享内存实现 CPU 集群上的完全批量训练，减小了通过最小顶点切割图分区算法通信的要求，使用一系列延迟更新算法避免了通信，并在 Reddit、OGB-Products、OGB-Papers 和 Proteins 等常见 GNN 基准数据集上获得了 3.7 倍到 97 倍的加速。