The task of predicting conversion rates (CVR) lies at the heart of online
advertising systems aiming to optimize bids to meet advertiser performance
requirements. Even with the recent rise of deep neural networks, these
predictions are often made by factorization machines (FM), especially in
commercial settings where inference latency is key. These models are trained
using the logistic regression framework on labeled tabular data formed from
past user activity that is relevant to the task at hand.
Many advertisers only care about click-attributed conversions. A major
challenge in training models that predict conversions-given-clicks comes from
data sparsity - clicks are rare, conversions attributed to clicks are even
rarer. However, mitigating sparsity by adding conversions that are not
click-attributed to the training set impairs model calibration. Since
calibration is critical to achieving advertiser goals, this is infeasible.
In this work we use the well-known idea of self-supervised pre-training, and
use an auxiliary auto-encoder model trained on all conversion events, both
click-attributed and not, as a feature extractor to enrich the main CVR
prediction model. Since the main model does not train on non click-attributed
conversions, this does not impair calibration. We adapt the basic
self-supervised pre-training idea to our online advertising setup by using a
loss function designed for tabular data, facilitating continual learning by
ensuring auto-encoder stability, and incorporating a neural network into a
large-scale real-time ad auction that ranks tens of thousands of ads, under
strict latency constraints, and without incurring a major engineering cost. We
show improvements both offline, during training, and in an online A/B test.
Following its success in A/B tests, our solution is now fully deployed to the
Yahoo native advertising system.

使用自监督预训练的方法来改善在线广告系统中的点击率预测模型，通过训练一个辅助自编码器模型来丰富主要的点击率预测模型，提高相关性、稳定性和网络规模，最终成功部署到 Yahoo 原生广告系统。

通过自监督预训练提升在线广告中的转化率预测

Improving conversion rate prediction via self-supervised pre-training in  online advertising

Predicting conversion rates (CVRs) in display advertising (e.g., predicting
the proportion of users who purchase an item (i.e., a conversion) after its
corresponding ad is clicked) is important when measuring the effects of ads
shown to users and to understanding the interests of the users. There is
generally a time delay (i.e., so-called {\it delayed feedback}) between the ad
click and conversion. Owing to the delayed feedback, samples that are converted
after an observation period may be treated as negative. To overcome this
drawback, CVR prediction assuming that the time delay follows an exponential
distribution has been proposed. In practice, however, there is no guarantee
that the delay is generated from the exponential distribution, and the best
distribution with which to represent the delay depends on the data. In this
paper, we propose a nonparametric delayed feedback model for CVR prediction
that represents the distribution of the time delay without assuming a
parametric distribution, such as an exponential or Weibull distribution.
Because the distribution of the time delay is modeled depending on the content
of an ad and the features of a user, various shapes of the distribution can be
represented potentially. In experiments, we show that the proposed model can
capture the distribution for the time delay on a synthetic dataset, even when
the distribution is complicated. Moreover, on a real dataset, we show that the
proposed model outperforms the existing method that assumes an exponential
distribution for the time delay in terms of conversion rate prediction.

本研究提出了一种非参数化滞后反馈模型，用于预测展示广告中的转化率，可以反映各种时间延迟的分布形状，并在实验中证明其优于指数分布模型。