Online field experiments are the gold-standard way of evaluating changes to
real-world interactive machine learning systems. Yet our ability to explore
complex, multi-dimensional policy spaces - such as those found in
recommendation and ranking problems - is often constrained by the limited
number of experiments that can be run simultaneously. To alleviate these
constraints, we augment online experiments with an offline simulator and apply
multi-task Bayesian optimization to tune live machine learning systems. We
describe practical issues that arise in these types of applications, including
biases that arise from using a simulator and assumptions for the multi-task
kernel. We measure empirical learning curves which show substantial gains from
including data from biased offline experiments, and show how these learning
curves are consistent with theoretical results for multi-task Gaussian process
generalization. We find that improved kernel inference is a significant driver
of multi-task generalization. Finally, we show several examples of Bayesian
optimization efficiently tuning a live machine learning system by combining
offline and online experiments.

使用离线模拟器并应用多任务贝叶斯优化改进在线机器学习系统的方法，较之仅进行在线实验，能够更有效地探索复杂、多维度的策略空间，并通过学习曲线表明离线实验可以显著提高在线实验结果的准确性和优化速度。