Recommendation systems are unique as they show a conflation of compute and
memory intensity due to their deep learning and massive embedding tables.
Training these models typically involve a hybrid CPU-GPU mode, where GPUs
accelerate the deep learning portion and the CPUs store and process the
memory-intensive embedding tables. The hybrid mode incurs a substantial
CPU-to-GPU transfer time and relies on main memory bandwidth to feed embeddings
to GPU for deep learning acceleration. Alternatively, we can store the entire
embeddings across GPUs to avoid the transfer time and utilize the GPU's High
Bandwidth Memory (HBM). This approach requires GPU-to-GPU backend communication
and scales the number of GPUs with the size of the embedding tables. To
overcome these concerns, this paper offers a heterogeneous acceleration
pipeline, called Hotline.
Hotline leverages the insight that only a small number of embedding entries
are accessed frequently, and can easily fit in a single GPU's HBM. Hotline
implements a data-aware and model-aware scheduling pipeline that utilizes the
(1) CPU main memory for not-frequently-accessed embeddings and (2) GPUs' local
memory for frequently-accessed embeddings. Hotline improves the training
throughput by dynamically stitching the execution of popular and not-popular
inputs through a novel hardware accelerator and feeding to the GPUs. Results on
real-world datasets and recommender models show that Hotline reduces the
average training time by 3x and 1.8x in comparison to Intel-optimized CPU-GPU
DLRM and HugeCTR-optimized GPU-only baseline, respectively. Hotline increases
the overall training throughput to 35.7 epochs/hour in comparison to 5.3
epochs/hour for the Intel-optimized DLRM baseline

Hotline 是一种数据感知和模型感知调度 pipeline，它通过实现一种动态的硬件加速器和馈送到 GPU 的方法，通过 GPU 加速减少训练时间，相比于现有的 CPU-GPU 混合模式，Hotline 平均训练时间降低了 3 倍，并且将整体训练吞吐量增加到每小时 35.7 个 epochs。