Graph neural networks (GNNs) are powerful tools for learning from graph data
and are widely used in various applications such as social network
recommendation, fraud detection, and graph search. The graphs in these
applications are typically large, usually containing hundreds of millions of
nodes. Training GNN models on such large graphs efficiently remains a big
challenge. Despite a number of sampling-based methods have been proposed to
enable mini-batch training on large graphs, these methods have not been proved
to work on truly industry-scale graphs, which require GPUs or mixed-CPU-GPU
training. The state-of-the-art sampling-based methods are usually not optimized
for these real-world hardware setups, in which data movement between CPUs and
GPUs is a bottleneck. To address this issue, we propose Global Neighborhood
Sampling that aims at training GNNs on giant graphs specifically for
mixed-CPU-GPU training. The algorithm samples a global cache of nodes
periodically for all mini-batches and stores them in GPUs. This global cache
allows in-GPU importance sampling of mini-batches, which drastically reduces
the number of nodes in a mini-batch, especially in the input layer, to reduce
data copy between CPU and GPU and mini-batch computation without compromising
the training convergence rate or model accuracy. We provide a highly efficient
implementation of this method and show that our implementation outperforms an
efficient node-wise neighbor sampling baseline by a factor of 2X-4X on giant
graphs. It outperforms an efficient implementation of LADIES with small layers
by a factor of 2X-14X while achieving much higher accuracy than LADIES.We also
theoretically analyze the proposed algorithm and show that with cached node
data of a proper size, it enjoys a comparable convergence rate as the
underlying node-wise sampling method.

这篇论文提出了一种全局节点采样算法，名为 Global Neighborhood Sampling，以满足大规模复杂图的图神经网络训练需求，该算法能使混合 CPU-GPU 训练的高效训练成为可能，并通过实验展现了比传统邻居采样方法更加高效的结果。