Deep learning techniques have been applied widely in industrial
recommendation systems. However, far less attention has been paid to the
overfitting problem of models in recommendation systems, which, on the
contrary, is recognized as a critical issue for deep neural networks. In the
context of Click-Through Rate (CTR) prediction, we observe an interesting
one-epoch overfitting problem: the model performance exhibits a dramatic
degradation at the beginning of the second epoch. Such a phenomenon has been
witnessed widely in real-world applications of CTR models. Thereby, the best
performance is usually achieved by training with only one epoch. To understand
the underlying factors behind the one-epoch phenomenon, we conduct extensive
experiments on the production data set collected from the display advertising
system of Alibaba. The results show that the model structure, the optimization
algorithm with a fast convergence rate, and the feature sparsity are closely
related to the one-epoch phenomenon. We also provide a likely hypothesis for
explaining such a phenomenon and conduct a set of proof-of-concept experiments.
We hope this work can shed light on future research on training more epochs for
better performance.

本研究表明，深度学习在工业推荐系统中得到了广泛的应用，但其模型在推荐系统中的过拟合问题受到的关注较少，而在点击率（CTR）预测中，出现了一个有趣的一次过拟合问题：模型性能在第二次迭代的开始时急剧下降，因此，通常使用仅一次迭代训练时性能最佳。本研究在阿里巴巴的广告系统数据集上进行了广泛的实验，结果表明，模型结构，具有快速收敛率的优化算法以及特征的稀疏性与一次过度拟合的现象密切相关，并提供了一个可能的假设来解释这种现象，并进行了一系列的概念证明实验，希望这项工作能为未来研究提供启示，训练更多迭代以获得更好的性能。

深度点击率预测模型过拟合现象的理解

Towards Understanding the Overfitting Phenomenon of Deep Click-Through Rate Prediction Models

This paper introduces the large scale visual search algorithm and system
infrastructure at Alibaba. The following challenges are discussed under the
E-commercial circumstance at Alibaba (a) how to handle heterogeneous image data
and bridge the gap between real-shot images from user query and the online
images. (b) how to deal with large scale indexing for massive updating data.
(c) how to train deep models for effective feature representation without huge
human annotations. (d) how to improve the user engagement by considering the
quality of the content. We take advantage of large image collection of Alibaba
and state-of-the-art deep learning techniques to perform visual search at
scale. We present solutions and implementation details to overcome those
problems and also share our learnings from building such a large scale
commercial visual search engine. Specifically, model and search-based fusion
approach is introduced to effectively predict categories. Also, we propose a
deep CNN model for joint detection and feature learning by mining user click
behavior. The binary index engine is designed to scale up indexing without
compromising recall and precision. Finally, we apply all the stages into an
end-to-end system architecture, which can simultaneously achieve highly
efficient and scalable performance adapting to real-shot images. Extensive
experiments demonstrate the advancement of each module in our system. We hope
visual search at Alibaba becomes more widely incorporated into today's
commercial applications.

本文介绍了阿里巴巴的大规模视觉搜索算法和系统架构，并探讨了在电商环境下如何处理异构的图像数据，在大数据更新中如何处理大规模索引，如何在无大量人工注释的情况下训练有效的深度模型，以及如何通过考虑内容质量来提高用户参与度，最后将所有阶段应用于一个端到端的系统架构，并展示了实验结果。