The real-world testing of decisions made using causal machine learning models
is an essential prerequisite for their successful application. We focus on
evaluating and improving contextual treatment assignment decisions: these are
personalised treatments applied to e.g. customers, each with their own
contextual information, with the aim of maximising a reward. In this paper we
introduce a model-agnostic framework for gathering data to evaluate and improve
contextual decision making through Bayesian Experimental Design. Specifically,
our method is used for the data-efficient evaluation of the regret of past
treatment assignments. Unlike approaches such as A/B testing, our method avoids
assigning treatments that are known to be highly sub-optimal, whilst engaging
in some exploration to gather pertinent information. We achieve this by
introducing an information-based design objective, which we optimise
end-to-end. Our method applies to discrete and continuous treatments. Comparing
our information-theoretic approach to baselines in several simulation studies
demonstrates the superior performance of our proposed approach.

本文提出了一种用于评估和改进因果机器学习模型下上下文治疗分配决策的数据采集框架，采用贝叶斯实验设计用于数据高效率评估和改进过去治疗分配的遗憾。与 A / B 测试等方法相比，我们的方法通过引入基于信息的设计目标来避免分配已知高度次优的治疗方法，同时进行探索以收集相关信息。我们的方法适用于离散和连续治疗，与其他基线相比，模拟研究表明了我们提出的信息理论方法具有更好的性能。