This paper studies policy evaluation with multiple data sources, especially
in scenarios that involve one experimental dataset with two arms, complemented
by a historical dataset generated under a single control arm. We propose novel
data integration methods that linearly integrate base policy value estimators
constructed based on the experimental and historical data, with weights
optimized to minimize the mean square error (MSE) of the resulting combined
estimator. We further apply the pessimistic principle to obtain more robust
estimators, and extend these developments to sequential decision making.
Theoretically, we establish non-asymptotic error bounds for the MSEs of our
proposed estimators, and derive their oracle, efficiency and robustness
properties across a broad spectrum of reward shift scenarios. Numerical
experiments and real-data-based analyses from a ridesharing company demonstrate
the superior performance of the proposed estimators.

本文研究了使用多个数据源进行政策评估的方法，特别是在一个包含两个实验群体的实验数据集，并补充了一个由单一对照群体生成的历史数据集的情景中。我们提出了一种新颖的数据集成方法，线性地整合基于实验数据和历史数据构建的基于政策值的估计器，通过优化权重以最小化结果估计器的均方误差（MSE）。我们进一步应用悲观原则来获得更加鲁棒的估计器，并将这些方法扩展到顺序决策制定。从理论上讲，我们建立了我们提出的估计器的非渐近误差界，并推导出在广泛的奖励变动情景下其适用性、效率和鲁棒性属性。来自一家共享出行公司的数值实验和基于真实数据的分析证明了所提出估计器的卓越性能。