Hybrid closed loop systems represent the future of care for people with type
1 diabetes (T1D). These devices usually utilise simple control algorithms to
select the optimal insulin dose for maintaining blood glucose levels within a
healthy range. Online reinforcement learning (RL) has been utilised as a method
for further enhancing glucose control in these devices. Previous approaches
have been shown to reduce patient risk and improve time spent in the target
range when compared to classical control algorithms, but are prone to
instability in the learning process, often resulting in the selection of unsafe
actions. This work presents an evaluation of offline RL as a means for
developing clinically effective dosing policies without the need for patient
interaction. This paper examines the utility of BCQ, CQL and TD3-BC in managing
the blood glucose of nine virtual patients within the UVA/Padova glucose
dynamics simulator. When trained on less than a tenth of the data required by
online RL approaches, this work shows that offline RL can significantly
increase time in the healthy blood glucose range when compared to the strongest
state-of-art baseline. This is achieved without any associated increase in low
blood glucose events. Offline RL is also shown to be able to correct for common
and challenging scenarios such as incorrect bolus dosing, irregular meal
timings and sub-optimal training data.

本文评估了离线强化学习方法，用于开发临床有效的药量策略，通过对 UVA/Padova 血糖动力学模拟器内的九个虚拟患者的血糖控制进行了研究，发现离线强化学习可以显著提高健康血糖范围内的时间，而不增加低血糖事件。