Systems for serving inference requests on graph neural networks (GNN) must combine low latency with high throughout, but they face irregular computation due to skew in the number of sampled graph nodes and aggregated GNN features. This makes it challenging to exploit GPUs effectively: using GPUs to sample only a few graph nodes yields lower performance than CPU-based sampling; and aggregating many features exhibits high data movement costs between GPUs and CPUs. Therefore, current GNN serving systems use CPUs for graph sampling and feature aggregation, limiting throughput. We describe Quiver, a distributed GPU-based GNN serving system with low-latency and high-throughput. Quiver's key idea is to exploit workload metrics for predicting the irregular computation of GNN requests, and governing the use of GPUs for graph sampling and feature aggregation: (1) for graph sampling, Quiver calculates the probabilistic sampled graph size, a metric that predicts the degree of parallelism in graph sampling. Quiver uses this metric to assign sampling tasks to GPUs only when the performance gains surpass CPU-based sampling; and (2) for feature aggregation, Quiver relies on the feature access probability to decide which features to partition and replicate across a distributed GPU NUMA topology. We show that Quiver achieves up to 35 times lower latency with an 8 times higher throughput compared to state-of-the-art GNN approaches (DGL and PyG).

本研究讨论了基于图神经网络(GNN)的推理系统如何结合低延迟和高吞吐量，提出了一个分布式GPU-based GNN服务系统 靠工作负载指标预测GNN请求的不规则计算特征并管理GPU的使用，利用概率性采样来提高GPU的采样性能并指导特征聚合。通过与现有的GNN方法（DGL和PyG）的比较，Quiver取得了高达35倍的低延迟和8倍的高吞吐量。

Quiver：支持工作负载感知的低延迟、高吞吐量 GNN 服务 GPU 支持