Problem definition: Behavioral health interventions, delivered through
digital platforms, have the potential to significantly improve health outcomes,
through education, motivation, reminders, and outreach. We study the problem of
optimizing personalized interventions for patients to maximize some long-term
outcome, in a setting where interventions are costly and capacity-constrained.
Methodology/results: This paper provides a model-free approach to solving
this problem. We find that generic model-free approaches from the reinforcement
learning literature are too data intensive for healthcare applications, while
simpler bandit approaches make progress at the expense of ignoring long-term
patient dynamics. We present a new algorithm we dub DecompPI that approximates
one step of policy iteration. Implementing DecompPI simply consists of a
prediction task from offline data, alleviating the need for online
experimentation. Theoretically, we show that under a natural set of structural
assumptions on patient dynamics, DecompPI surprisingly recovers at least 1/2 of
the improvement possible between a naive baseline policy and the optimal
policy. At the same time, DecompPI is both robust to estimation errors and
interpretable. Through an empirical case study on a mobile health platform for
improving treatment adherence for tuberculosis, we find that DecompPI can
provide the same efficacy as the status quo with approximately half the
capacity of interventions.
Managerial implications: DecompPI is general and is easily implementable for
organizations aiming to improve long-term behavior through targeted
interventions. Our case study suggests that the platform's costs of deploying
interventions can potentially be cut by 50%, which facilitates the ability to
scale up the system in a cost-efficient fashion.

本文提出了一种名为 DecompPI 的新算法，用于优化基于行为干预的个性化健康干预计划，以实现卓越的长期健康成果；我们发现，该算法不仅具有一定的理论保证，而且在现实应用中也能够显著地减少干预成本。