This paper studies policy evaluation with multiple data sources, especially
in scenarios that involve one experimental dataset with two arms, complemented
by a historical dataset generated under a single control arm. We propose novel
data integration methods that linearly integrate base policy value estimators
constructed based on the experimental and historical data, with weights
optimized to minimize the mean square error (MSE) of the resulting combined
estimator. We further apply the pessimistic principle to obtain more robust
estimators, and extend these developments to sequential decision making.
Theoretically, we establish non-asymptotic error bounds for the MSEs of our
proposed estimators, and derive their oracle, efficiency and robustness
properties across a broad spectrum of reward shift scenarios. Numerical
experiments and real-data-based analyses from a ridesharing company demonstrate
the superior performance of the proposed estimators.

本文研究了使用多个数据源进行政策评估的方法，特别是在一个包含两个实验群体的实验数据集，并补充了一个由单一对照群体生成的历史数据集的情景中。我们提出了一种新颖的数据集成方法，线性地整合基于实验数据和历史数据构建的基于政策值的估计器，通过优化权重以最小化结果估计器的均方误差（MSE）。我们进一步应用悲观原则来获得更加鲁棒的估计器，并将这些方法扩展到顺序决策制定。从理论上讲，我们建立了我们提出的估计器的非渐近误差界，并推导出在广泛的奖励变动情景下其适用性、效率和鲁棒性属性。来自一家共享出行公司的数值实验和基于真实数据的分析证明了所提出估计器的卓越性能。

结合实验和历史数据进行政策评估

Combining Experimental and Historical Data for Policy Evaluation

We present a data-driven control architecture for modifying the kinematics of
robots and artificial avatars to encode specific information such as the
presence or not of an emotion in the movements of an avatar or robot driven by
a human operator. We validate our approach on an experimental dataset obtained
during the reach-to-grasp phase of a pick-and-place task.

我们提出了一个基于数据驱动的控制架构，用于修改机器人和人工化身的运动学，以编码由人操作驱动的化身或机器人运动中的特定信息，例如情感的存在与否。我们在一个取放任务的夹取阶段的实验数据集上验证了我们的方法。