Deep Click-Through Rate (CTR) prediction models play an important role in
modern industrial recommendation scenarios. However, high memory overhead and
computational costs limit their deployment in resource-constrained
environments. Low-rank approximation is an effective method for computer vision
and natural language processing models, but its application in compressing CTR
prediction models has been less explored. Due to the limited memory and
computing resources, compression of CTR prediction models often confronts three
fundamental challenges, i.e., (1). How to reduce the model sizes to adapt to
edge devices? (2). How to speed up CTR prediction model inference? (3). How to
retain the capabilities of original models after compression? Previous low-rank
compression research mostly uses tensor decomposition, which can achieve a high
parameter compression ratio, but brings in AUC degradation and additional
computing overhead. To address these challenges, we propose a unified low-rank
decomposition framework for compressing CTR prediction models. We find that
even with the most classic matrix decomposition SVD method, our framework can
achieve better performance than the original model. To further improve the
effectiveness of our framework, we locally compress the output features instead
of compressing the model weights. Our unified low-rank compression framework
can be applied to embedding tables and MLP layers in various CTR prediction
models. Extensive experiments on two academic datasets and one real industrial
benchmark demonstrate that, with 3-5x model size reduction, our compressed
models can achieve both faster inference and higher AUC than the uncompressed
original models. Our code is at
this https URL

通过低秩压缩模型，本研究提出了一种统一的压缩 CTR 预测模型的方法，实现了模型尺寸减小、推理速度加快、AUC 提高等优势。