We introduce a distributionally robust approach that enhances the reliability of offline policy evaluation in contextual bandits under general covariate shifts. Our method aims to deliver robust policy evaluation results in the presence of discrepancies in both context and policy distribution between logging and target data. Central to our methodology is the application of robust regression, a distributionally robust technique tailored here to improve the estimation of conditional reward distribution from logging data. Utilizing the reward model obtained from robust regression, we develop a comprehensive suite of policy value estimators, by integrating our reward model into established evaluation frameworks, namely direct methods and doubly robust methods. Through theoretical analysis, we further establish that the proposed policy value estimators offer a finite sample upper bound for the bias, providing a clear advantage over traditional methods, especially when the shift is large. Finally, we designed an extensive range of policy evaluation scenarios, covering diverse magnitudes of shifts and a spectrum of logging and target policies. Our empirical results indicate that our approach significantly outperforms baseline methods, most notably in 90% of the cases under the policy shift-only settings and 72% of the scenarios under the general covariate shift settings.

我们介绍了一种分布健壮的方法，用于在背景变量移位下增强上下文赌博的离线策略评估的可靠性。通过应用分布健壮回归技术改进条件奖励分布的估计，我们开发出一套综合的策略价值评估器，并通过理论分析证明了该方法相对于传统方法在偏移较大时的有限样本上限优势。在广泛的策略评估场景中，我们的实证结果表明我们的方法明显优于基准方法。

上下文强化学习中基于一般协变量转移的分布稳健策略评估