Online reinforcement learning (RL) algorithms offer great potential for
personalizing treatment for participants in clinical trials. However, deploying
an online, autonomous algorithm in the high-stakes healthcare setting makes
quality control and data quality especially difficult to achieve. This paper
proposes algorithm fidelity as a critical requirement for deploying online RL
algorithms in clinical trials. It emphasizes the responsibility of the
algorithm to (1) safeguard participants and (2) preserve the scientific utility
of the data for post-trial analyses. We also present a framework for
pre-deployment planning and real-time monitoring to help algorithm developers
and clinical researchers ensure algorithm fidelity. To illustrate our
framework's practical application, we present real-world examples from the
Oralytics clinical trial. Since Spring 2023, this trial successfully deployed
an autonomous, online RL algorithm to personalize behavioral interventions for
participants at risk for dental disease.

在线强化学习算法在临床试验中为参与者个性化治疗提供了巨大潜力，然而，在高风险医疗环境中部署在线自主算法使得质量控制和数据质量尤其困难。本文提出了算法保真度作为临床试验中部署在线强化学习算法的关键要求，并强调算法对于 (1) 保护参与者和 (2) 保留数据进行试后分析的科学效用具有责任。我们还提出了一个用于部署前规划和实时监控的框架，以帮助算法开发者和临床研究人员确保算法保真度。为了说明我们框架的实际应用，我们提供了来自于 Oralytics 临床试验的真实案例。自 2023 年春季以来，该试验成功地部署了一种自主的在线强化学习算法，用于个性化行为干预以防止牙疾的风险参与者。