Calibration is defined as the ratio of the average predicted click rate to
the true click rate. The optimization of calibration is essential to many
online advertising recommendation systems because it directly affects the
downstream bids in ads auctions and the amount of money charged to advertisers.
Despite its importance, calibration optimization often suffers from a problem
called "maximization bias". Maximization bias refers to the phenomenon that the
maximum of predicted values overestimates the true maximum. The problem is
introduced because the calibration is computed on the set selected by the
prediction model itself. It persists even if unbiased predictions can be
achieved on every datapoint and worsens when covariate shifts exist between the
training and test sets. To mitigate this problem, we theorize the
quantification of maximization bias and propose a variance-adjusting debiasing
(VAD) meta-algorithm in this paper. The algorithm is efficient, robust, and
practical as it is able to mitigate maximization bias problems under covariate
shifts, neither incurring additional online serving costs nor compromising the
ranking performance. We demonstrate the effectiveness of the proposed algorithm
using a state-of-the-art recommendation neural network model on a large-scale
real-world dataset.

该论文提出了一种方案解决优化校准问题中的最大化偏差问题，即使用一种新的算法降低最大化偏差的影响，实验证明该算法对于协变量漂移的问题很有效，而且不会影响机器学习模型的性能表现。

校准的重要性：解决大规模广告推荐系统中的最大化偏差问题

Calibration Matters: Tackling Maximization Bias in Large-scale  Advertising Recommendation Systems

The Q-learning algorithm is known to be affected by the maximization bias,
i.e. the systematic overestimation of action values, an important issue that
has recently received renewed attention. Double Q-learning has been proposed as
an efficient algorithm to mitigate this bias. However, this comes at the price
of an underestimation of action values, in addition to increased memory
requirements and a slower convergence. In this paper, we introduce a new way to
address the maximization bias in the form of a "self-correcting algorithm" for
approximating the maximum of an expected value. Our method balances the
overestimation of the single estimator used in conventional Q-learning and the
underestimation of the double estimator used in Double Q-learning. Applying
this strategy to Q-learning results in Self-correcting Q-learning. We show
theoretically that this new algorithm enjoys the same convergence guarantees as
Q-learning while being more accurate. Empirically, it performs better than
Double Q-learning in domains with rewards of high variance, and it even attains
faster convergence than Q-learning in domains with rewards of zero or low
variance. These advantages transfer to a Deep Q Network implementation that we
call Self-correcting DQN and which outperforms regular DQN and Double DQN on
several tasks in the Atari 2600 domain.

本文介绍了一种新的自我校正 Q 学习算法，其通过平衡常规 Q-learning 中使用的单估计器的过高估计和 Double Q-learning 中使用的双估计器的低估计来解决最大化偏差问题，并提出了应用于 Deep Q Network 的 Self-correcting DQN 实现，在 Atari 2600 领域的多项任务中表现更好