Predictive models are often introduced to decision-making tasks under the
rationale that they improve performance over an existing decision-making
policy. However, it is challenging to compare predictive performance against an
existing decision-making policy that is generally under-specified and dependent
on unobservable factors. These sources of uncertainty are often addressed in
practice by making strong assumptions about the data-generating mechanism. In
this work, we propose a method to compare the predictive performance of
decision policies under a variety of modern identification approaches from the
causal inference and off-policy evaluation literatures (e.g., instrumental
variable, marginal sensitivity model, proximal variable). Key to our method is
the insight that there are regions of uncertainty that we can safely ignore in
the policy comparison. We develop a practical approach for finite-sample
estimation of regret intervals under no assumptions on the parametric form of
the status quo policy. We verify our framework theoretically and via synthetic
data experiments. We conclude with a real-world application using our framework
to support a pre-deployment evaluation of a proposed modification to a
healthcare enrollment policy.

我们提出了一种方法来比较决策策略的预测性能，该方法结合了因果推断和离线评估文献中的现代识别方法，并使用我们的框架在真实世界的医疗保险注册政策的预部署评估中进行了应用。