In this talk, we introduce Merlin HugeCTR. Merlin HugeCTR is an open source,
GPU-accelerated integration framework for click-through rate estimation. It
optimizes both training and inference, whilst enabling model training at scale
with model-parallel embeddings and data-parallel neural networks. In
particular, Merlin HugeCTR combines a high-performance GPU embedding cache with
an hierarchical storage architecture, to realize low-latency retrieval of
embeddings for online model inference tasks. In the MLPerf v1.0 DLRM model
training benchmark, Merlin HugeCTR achieves a speedup of up to 24.6x on a
single DGX A100 (8x A100) over PyTorch on 4x4-socket CPU nodes (4x4x28 cores).
Merlin HugeCTR can also take advantage of multi-node environments to accelerate
training even further. Since late 2021, Merlin HugeCTR additionally features a
hierarchical parameter server (HPS) and supports deployment via the NVIDIA
Triton server framework, to leverage the computational capabilities of GPUs for
high-speed recommendation model inference. Using this HPS, Merlin HugeCTR users
can achieve a 5~62x speedup (batch size dependent) for popular recommendation
models over CPU baseline implementations, and dramatically reduce their
end-to-end inference latency.

Merlin HugeCTR 是一种用于 CTR 估算的开源 GPU 加速集成框架，支持训练和推断优化，利用模型并行嵌入和数据并行神经网络进行规模化的模型训练，并使用高性能 GPU 嵌入式缓存和分层存储架构相结合，以实现低延迟的在线推荐任务。支持使用 NVIDIA Triton 服务器框架部署，可大幅减少端到端推荐模型推断的延迟。