This paper describes a new method for representing embedding tables of graph
neural networks (GNNs) more compactly via tensor-train (TT) decomposition. We
consider the scenario where (a) the graph data that lack node features, thereby
requiring the learning of embeddings during training; and (b) we wish to
exploit GPU platforms, where smaller tables are needed to reduce host-to-GPU
communication even for large-memory GPUs. The use of TT enables a compact
parameterization of the embedding, rendering it small enough to fit entirely on
modern GPUs even for massive graphs. When combined with judicious schemes for
initialization and hierarchical graph partitioning, this approach can reduce
the size of node embedding vectors by 1,659 times to 81,362 times on large
publicly available benchmark datasets, achieving comparable or better accuracy
and significant speedups on multi-GPU systems. In some cases, our model without
explicit node features on input can even match the accuracy of models that use
node features.

本文利用张量列车分解的方法，将图神经网络的嵌入表更紧凑地表示，从而降低了模型大小，实现了在大型公共基准数据集上节点嵌入向量大小的减小，且在多 GPU 系统上实现了相当或更好的准确性和显著的加速。本文还介绍了用于降低主机到 GPU 通信的方案。

使用 Tensor-Train 分解提升 Nimble GNN 嵌入性能

Nimble GNN Embedding with Tensor-Train Decomposition

As the emerging trend of graph-based deep learning, Graph Neural Networks
(GNNs) excel for their capability to generate high-quality node feature vectors
(embeddings). However, the existing one-size-fits-all GNN implementations are
insufficient to catch up with the evolving GNN architectures, the
ever-increasing graph sizes, and the diverse node embedding dimensionalities.
To this end, we propose \textbf{GNNAdvisor}, an adaptive and efficient runtime
system to accelerate various GNN workloads on GPU platforms. First, GNNAdvisor
explores and identifies several performance-relevant features from both the GNN
model and the input graph, and uses them as a new driving force for GNN
acceleration. Second, GNNAdvisor implements a novel and highly-efficient 2D
workload management, tailored for GNN computation to improve GPU utilization
and performance under different application settings. Third, GNNAdvisor
capitalizes on the GPU memory hierarchy for acceleration by gracefully
coordinating the execution of GNNs according to the characteristics of the GPU
memory structure and GNN workloads. Furthermore, to enable automatic runtime
optimization, GNNAdvisor incorporates a lightweight analytical model for an
effective design parameter search. Extensive experiments show that GNNAdvisor
outperforms the state-of-the-art GNN computing frameworks, such as Deep Graph
Library ($3.02\times$ faster on average) and NeuGraph (up to $4.10\times$
faster), on mainstream GNN architectures across various datasets.

GNNAdvisor 是一种自适应、高效的运行时系统，旨在加速 GPU 平台上各种 GNN 工作负载，使用了新的驱动力、二维工作负载管理和 GPU 内存层次优化等技术，实现了自动运行时优化。