In this paper, we present an industry ad recommendation system, paying
attention to the challenges and practices of learning appropriate
representations. Our study begins by showcasing our approaches to preserving
priors when encoding features of diverse types into embedding representations.
Specifically, we address sequence features, numeric features, pre-trained
embedding features, as well as sparse ID features. Moreover, we delve into two
pivotal challenges associated with feature representation: the dimensional
collapse of embeddings and the interest entanglement across various tasks or
scenarios. Subsequently, we propose several practical approaches to effectively
tackle these two challenges. We then explore several training techniques to
facilitate model optimization, reduce bias, and enhance exploration.
Furthermore, we introduce three analysis tools that enable us to
comprehensively study feature correlation, dimensional collapse, and interest
entanglement. This work builds upon the continuous efforts of Tencent's ads
recommendation team in the last decade. It not only summarizes general design
principles but also presents a series of off-the-shelf solutions and analysis
tools. The reported performance is based on our online advertising platform,
which handles hundreds of billions of requests daily, serving millions of ads
to billions of users.

本文介绍了一种行业广告推荐系统，重点关注学习合适表示的挑战和实践。我们展示了保留先验信息的方法，将各种类型的特征编码为嵌入表示。此外，我们探讨了特征表示中的两个关键挑战：嵌入维度崩塌和各种任务或场景的兴趣交织。随后，我们提出了几种实用的方法来有效应对这两个挑战。我们还探索了几种训练技术，以促进模型优化、减少偏差和增强探索。此外，我们介绍了三种分析工具，可以全面研究特征相关性、维度崩塌和兴趣交织。本研究建立在腾讯广告推荐团队过去十年的持续努力基础上，不仅总结了一般的设计原则，还提供了一系列现成的解决方案和分析工具。所报告的性能基于我们的在线广告平台，每天处理数千亿个请求，为数十亿用户提供数百万个广告。