Deep learning recommendation models (DLRMs) have been widely applied in
Internet companies. The embedding tables of DLRMs are too large to fit on GPU
memory entirely. We propose a GPU-based software cache approaches to
dynamically manage the embedding table in the CPU and GPU memory space by
leveraging the id's frequency statistics of the target dataset. Our proposed
software cache is efficient in training entire DLRMs on GPU in a synchronized
update manner. It is also scaled to multiple GPUs in combination with the
widely used hybrid parallel training approaches. Evaluating our prototype
system shows that we can keep only 1.5% of the embedding parameters in the GPU
to obtain a decent end-to-end training speed.

本文提出了一种基于 GPU 的软件缓存方法来动态管理 DLRM 的 embedding table，通过利用目标数据集 id 的频率统计信息，在 GPU 和 CPU 内存空间之间进行管理，以此来使得在 GPU 上训练 DLRM 更为实用。同时，该方法也支持多 GPU 并行训练，通过实验证明，只在 GPU 上保留 1.5% 的 embedding 参数就能保障训练速度。