Dental disease is one of the most common chronic diseases despite being
largely preventable. However, professional advice on optimal oral hygiene
practices is often forgotten or abandoned by patients. Therefore patients may
benefit from timely and personalized encouragement to engage in oral self-care
behaviors. In this paper, we develop an online reinforcement learning (RL)
algorithm for use in optimizing the delivery of mobile-based prompts to
encourage oral hygiene behaviors. One of the main challenges in developing such
an algorithm is ensuring that the algorithm considers the impact of the current
action on the effectiveness of future actions (i.e., delayed effects),
especially when the algorithm has been made simple in order to run stably and
autonomously in a constrained, real-world setting (i.e., highly noisy, sparse
data). We address this challenge by designing a quality reward which maximizes
the desired health outcome (i.e., high-quality brushing) while minimizing user
burden. We also highlight a procedure for optimizing the hyperparameters of the
reward by building a simulation environment test bed and evaluating candidates
using the test bed. The RL algorithm discussed in this paper will be deployed
in Oralytics, an oral self-care app that provides behavioral strategies to
boost patient engagement in oral hygiene practices.

本文开发了基于强化学习的算法，用于优化移动提示的发送，以鼓励口腔自我护理行为。研究致力于解决算法中的延迟效应问题，通过设计质量奖励并优化超参数选择，最终被部署在提供行为策略以提高口腔健康的 Oralytics 应用程序上。