Recommender models are commonly used to suggest relevant items to a user for
e-commerce and online advertisement-based applications. These models use
massive embedding tables to store numerical representation of items' and users'
categorical variables (memory intensive) and employ neural networks (compute
intensive) to generate final recommendations. Training these large-scale
recommendation models is evolving to require increasing data and compute
resources. The highly parallel neural networks portion of these models can
benefit from GPU acceleration however, large embedding tables often cannot fit
in the limited-capacity GPU device memory. Hence, this paper deep dives into
the semantics of training data and obtains insights about the feature access,
transfer, and usage patterns of these models. We observe that, due to the
popularity of certain inputs, the accesses to the embeddings are highly skewed
with a few embedding entries being accessed up to 10000x more. This paper
leverages this asymmetrical access pattern to offer a framework, called FAE,
and proposes a hot-embedding aware data layout for training recommender models.
This layout utilizes the scarce GPU memory for storing the highly accessed
embeddings, thus reduces the data transfers from CPU to GPU. At the same time,
FAE engages the GPU to accelerate the executions of these hot embedding
entries. Experiments on production-scale recommendation models with real
datasets show that FAE reduces the overall training time by 2.3x and 1.52x in
comparison to XDL CPU-only and XDL CPU-GPU execution while maintaining baseline
accuracy

通过研究推荐数据的使用方式和特征，本文提出了一种称为 FAE 框架的热嵌入感知数据布局，利用有限的 GPU 内存存储高访问嵌入，以减少 CPU 到 GPU 的数据传输并加速执行，从而将推荐模型的训练时间降低了 2.3 倍。