Training recommendation models pose significant challenges regarding resource
utilization and performance. Prior research has proposed an approach that
categorizes embeddings into popular and non-popular classes to reduce the
training time for recommendation models. We observe that, even among the
popular embeddings, certain embeddings undergo rapid training and exhibit
minimal subsequent variation, resulting in saturation. Consequently, updates to
these embeddings lack any contribution to model quality. This paper presents
Slipstream, a software framework that identifies stale embeddings on the fly
and skips their updates to enhance performance. This capability enables
Slipstream to achieve substantial speedup, optimize CPU-GPU bandwidth usage,
and eliminate unnecessary memory access. SlipStream showcases training time
reductions of 2x, 2.4x, 1.2x, and 1.175x across real-world datasets and
configurations, compared to Baseline XDL, Intel-optimized DRLM, FAE, and
Hotline, respectively.

Slipstream 是一个软件框架，通过识别过时的嵌入并跳过其更新来提高性能，在真实世界的数据集和配置方面，相比 Baseline XDL、Intel-optimized DRLM、FAE 和 Hotline，SlipStream 分别展示了 2 倍、2.4 倍、1.2 倍和 1.175 倍的训练时间缩短。