With the development of the multi-media internet, visual characteristics have
become an important factor affecting user interests. Thus, incorporating visual
features is a promising direction for further performance improvements in
click-through rate (CTR) prediction. However, we found that simply injecting
the image embeddings trained with established pre-training methods only has
marginal improvements. We attribute the failure to two reasons: First, The
pre-training methods are designed for well-defined computer vision tasks
concentrating on semantic features, and they cannot learn personalized interest
in recommendations. Secondly, pre-trained image embeddings only containing
semantic information have little information gain, considering we already have
semantic features such as categories and item titles as inputs in the CTR
prediction task. We argue that a pre-training method tailored for
recommendation is necessary for further improvements. To this end, we propose a
recommendation-aware image pre-training method that can learn visual features
from user click histories. Specifically, we propose a user interest
reconstruction module to mine visual features related to user interests from
behavior histories. We further propose a contrastive training method to avoid
collapsing of embedding vectors. We conduct extensive experiments to verify
that our method can learn users' visual interests, and our method achieves
$0.46\%$ improvement in offline AUC and $0.88\%$ improvement in Taobao online
GMV with p-value$<0.01$.

为提高 CTR 预测效果，本文提出了一种针对推荐领域的图像预训练方法，通过用户点击历史来学习用户的视觉兴趣，同时采用对比训练方法以避免嵌入向量的折叠，实验表明该方法可以提高 0.46% 的离线 AUC 和 0.88% 的淘宝线上 GMV。