We study decision making in environments where the reward is only partially
observed, but can be modeled as a function of an action and an observed
context. This setting, known as contextual bandits, encompasses a wide variety
of applications including health-care policy and Internet advertising. A
central task is evaluation of a new policy given historic data consisting of
contexts, actions and received rewards. The key challenge is that the past data
typically does not faithfully represent proportions of actions taken by a new
policy. Previous approaches rely either on models of rewards or models of the
past policy. The former are plagued by a large bias whereas the latter have a
large variance.
In this work, we leverage the strength and overcome the weaknesses of the two
approaches by applying the doubly robust technique to the problems of policy
evaluation and optimization. We prove that this approach yields accurate value
estimates when we have either a good (but not necessarily consistent) model of
rewards or a good (but not necessarily consistent) model of past policy.
Extensive empirical comparison demonstrates that the doubly robust approach
uniformly improves over existing techniques, achieving both lower variance in
value estimation and better policies. As such, we expect the doubly robust
approach to become common practice.

在具有上下文情境和目标函数的决策环境中，我们使用双重稳健技术评估新策略，并证明这种方法使价值估计具有较低的方差，且能达到更好的策略，为该领域提供一种有效的方法。