Recommender systems are essential for content-sharing platforms by curating
personalized content. To evaluate updates of recommender systems targeting
content creators, platforms frequently engage in creator-side randomized
experiments to estimate treatment effect, defined as the difference in outcomes
when a new (vs. the status quo) algorithm is deployed on the platform. We show
that the standard difference-in-means estimator can lead to a biased treatment
effect estimate. This bias arises because of recommender interference, which
occurs when treated and control creators compete for exposure through the
recommender system. We propose a "recommender choice model" that captures how
an item is chosen among a pool comprised of both treated and control content
items. By combining a structural choice model with neural networks, the
framework directly models the interference pathway in a microfounded way while
accounting for rich viewer-content heterogeneity. Using the model, we construct
a double/debiased estimator of the treatment effect that is consistent and
asymptotically normal. We demonstrate its empirical performance with a field
experiment on Weixin short-video platform: besides the standard creator-side
experiment, we carry out a costly blocked double-sided randomization design to
obtain a benchmark estimate without interference bias. We show that the
proposed estimator significantly reduces the bias in treatment effect estimates
compared to the standard difference-in-means estimator.

推荐系统对于内容分享平台的个性化内容整理至关重要。 通过进行创作者方面的随机实验以估计治疗效应，定义为当新的（而不是现状）算法在平台上部署时，平台经常评估面向内容创作者的推荐系统更新。我们展示了标准差异估计量可能导致偏倚的治疗效应估计。这种偏倚的产生是因为推荐干扰，即在推荐系统中，受治疗群体和对照群体的创作者竞争曝光机会。我们提出了一种 “推荐选择模型”，该模型捕捉到在包含受治疗和对照内容项的池中如何选择项目。通过结合结构性选择模型和神经网络，该框架以微观基础的方式直接建模推荐干扰路径，并考虑丰富的观众 - 内容异质性。使用该模型，我们构建了一种一致且渐近正态的治疗效应的双重 / 无偏估计量。我们通过对微信短视频平台进行的一项实地实验展示了其实证表现：除了标准的创作者方面实验外，我们还进行了昂贵的阻塞性双边随机化设计以获得没有干扰偏倚的基准估计。我们展示了相对于标准差异估计量，所提出的估计量显著减少了治疗效应估计中的偏倚。