The purpose of modeling document relevance for search engines is to rank
better in subsequent searches. Document-specific historical click-through rates
can be important features in a dynamic ranking system which updates as we
accumulate more sample. This paper describes the properties of several such
features, and tests them in controlled experiments. Extending the inverse
propensity weighting method to documents creates an unbiased estimate of
document relevance. This feature can approximate relevance accurately, leading
to near-optimal ranking in ideal circumstances. However, it has high variance
that is increasing with respect to the degree of position bias. Furthermore,
inaccurate position bias estimation leads to poor performance. Under several
scenarios this feature can perform worse than biased click-through rates. This
paper underscores the need for accurate position bias estimation, and is unique
in suggesting simultaneous use of biased and unbiased position bias features.

本文介绍了排名系统中模拟文档相关性的目的，并测试了几个特征的性质。将逆倾向权重方法用于文档可创建不偏的文档相关性估计特征，该特征可以准确地近似相关性，在理想情况下实现接近最佳的排名。然而，此特征具有较高的方差，且方差随着位置偏差的增加而增加。不准确的位置偏差估计导致性能下降。本文强调了准确估计位置偏差的必要性，并独特地提出了同时使用有偏和无偏位置偏差特征的建议。