Type 1 diabetes mellitus (T1D) is characterized by insulin deficiency and
blood glucose (BG) control issues. The state-of-the-art solution for continuous
BG control is reinforcement learning (RL), where an agent can dynamically
adjust exogenous insulin doses in time to maintain BG levels within the target
range. However, due to the lack of action guidance, the agent often needs to
learn from randomized trials to understand misleading correlations between
exogenous insulin doses and BG levels, which can lead to instability and
unsafety. To address these challenges, we propose an introspective RL based on
Counterfactual Invertible Neural Networks (CINN). We use the pre-trained CINN
as a frozen introspective block of the RL agent, which integrates forward
prediction and counterfactual inference to guide the policy updates, promoting
more stable and safer BG control. Constructed based on interpretable causal
order, CINN employs bidirectional encoders with affine coupling layers to
ensure invertibility while using orthogonal weight normalization to enhance the
trainability, thereby ensuring the bidirectional differentiability of network
parameters. We experimentally validate the accuracy and generalization ability
of the pre-trained CINN in BG prediction and counterfactual inference for
action. Furthermore, our experimental results highlight the effectiveness of
pre-trained CINN in guiding RL policy updates for more accurate and safer BG
control.

基于反事实可逆神经网络的内省强化学习在血糖控制中表现出更高的稳定性和安全性。