The escalating prevalence of cannabis use, and associated cannabis-use
disorder (CUD), poses a significant public health challenge globally. With a
notably wide treatment gap, especially among emerging adults (EAs; ages 18-25),
addressing cannabis use and CUD remains a pivotal objective within the 2030
United Nations Agenda for Sustainable Development Goals (SDG). In this work, we
develop an online reinforcement learning (RL) algorithm called reBandit which
will be utilized in a mobile health study to deliver personalized mobile health
interventions aimed at reducing cannabis use among EAs. reBandit utilizes
random effects and informative Bayesian priors to learn quickly and efficiently
in noisy mobile health environments. Moreover, reBandit employs Empirical Bayes
and optimization techniques to autonomously update its hyper-parameters online.
To evaluate the performance of our algorithm, we construct a simulation testbed
using data from a prior study, and compare against commonly used algorithms in
mobile health studies. We show that reBandit performs equally well or better
than all the baseline algorithms, and the performance gap widens as population
heterogeneity increases in the simulation environment, proving its adeptness to
adapt to diverse population of study participants.

使用名为 reBandit 的在线增强学习算法，以在移动健康研究中提供个性化的移动健康干预措施，旨在减少 18-25 岁的青年对大麻的使用。reBandit 在噪声较大的移动健康环境中利用随机效应和信息贝叶斯先验快速高效地进行学习，同时通过经验贝叶斯和优化技术在线自动更新超参数，证明其适应不同人群的能力优于其他算法。

reBandit：基于随机效应的在线强化学习算法用于减少大麻使用

reBandit: Random Effects based Online RL algorithm for Reducing Cannabis  Use

Artificial Intelligence and digital health have the potential to transform
global health. However, having access to representative data to test and
validate algorithms in realistic production environments is essential. We
introduce HealthSyn, an open-source synthetic data generator of user behavior
for testing reinforcement learning algorithms in the context of mobile health
interventions. The generator utilizes Markov processes to generate diverse user
actions, with individual user behavioral patterns that can change in reaction
to personalized interventions (i.e., reminders, recommendations, and
incentives). These actions are translated into actual logs using an ML-purposed
data schema specific to the mobile health application functionality included
with HealthKit, and open-source SDK. The logs can be fed to pipelines to obtain
user metrics. The generated data, which is based on real-world behaviors and
simulation techniques, can be used to develop, test, and evaluate, both ML
algorithms in research and end-to-end operational RL-based intervention
delivery frameworks.

使用 Markov 过程生成多样化的用户行为，以测试针对移动健康干预的强化学习算法的健康合成开源合成数据生成器 HealthSyn，可生成基于真实行为和模拟技术的数据，用于开发、测试和评估研究中的 ML 算法和端到端操作 RL 基础干预交付框架。

全球卫生中自适应干预的合成数据生成器

Synthetic Data Generator for Adaptive Interventions in Global Health

Due to the recent advancements in wearables and sensing technology, health
scientists are increasingly developing mobile health (mHealth) interventions.
In mHealth interventions, mobile devices are used to deliver treatment to
individuals as they go about their daily lives. These treatments are generally
designed to impact a near time, proximal outcome such as stress or physical
activity. The mHealth intervention policies, often called just-in-time adaptive
interventions, are decision rules that map an individual's current state (e.g.,
individual's past behaviors as well as current observations of time, location,
social activity, stress and urges to smoke) to a particular treatment at each
of many time points. The vast majority of current mHealth interventions deploy
expert-derived policies. In this paper, we provide an approach for conducting
inference about the performance of one or more such policies using historical
data collected under a possibly different policy. Our measure of performance is
the average of proximal outcomes over a long time period should the particular
mHealth policy be followed. We provide an estimator as well as confidence
intervals. This work is motivated by HeartSteps, an mHealth physical activity
intervention.

本文探讨了移动健康干预技术 (mHealth) 中的即时自适应干预政策，包括专家提供的政策以及基于历史数据的干预政策效果估计，并提供了一个估计器和置信区间，以评估这类政策在长期内是否能提高健康结果。