There is a growing interest in using reinforcement learning (RL) to
personalize sequences of treatments in digital health to support users in
adopting healthier behaviors. Such sequential decision-making problems involve
decisions about when to treat and how to treat based on the user's context
(e.g., prior activity level, location, etc.). Online RL is a promising
data-driven approach for this problem as it learns based on each user's
historical responses and uses that knowledge to personalize these decisions.
However, to decide whether the RL algorithm should be included in an
``optimized'' intervention for real-world deployment, we must assess the data
evidence indicating that the RL algorithm is actually personalizing the
treatments to its users. Due to the stochasticity in the RL algorithm, one may
get a false impression that it is learning in certain states and using this
learning to provide specific treatments. We use a working definition of
personalization and introduce a resampling-based methodology for investigating
whether the personalization exhibited by the RL algorithm is an artifact of the
RL algorithm stochasticity. We illustrate our methodology with a case study by
analyzing the data from a physical activity clinical trial called HeartSteps,
which included the use of an online RL algorithm. We demonstrate how our
approach enhances data-driven truth-in-advertising of algorithm personalization
both across all users as well as within specific users in the study.

该论文介绍了一种基于强化学习和在线学习的数据驱动方法，可以实现用户个性化健康治疗序列的优化。通过案例研究，论文使用重新取样方法对算法个性化进行评估，提高了算法的真实性，对于真实应用也具有参考价值。