In contemporary radiotherapy planning (RTP), a key module leaf sequencing is
predominantly addressed by optimization-based approaches. In this paper, we
propose a novel deep reinforcement learning (DRL) model termed as Reinforced
Leaf Sequencer (RLS) in a multi-agent framework for leaf sequencing. The RLS
model offers improvements to time-consuming iterative optimization steps via
large-scale training and can control movement patterns through the design of
reward mechanisms. We have conducted experiments on four datasets with four
metrics and compared our model with a leading optimization sequencer. Our
findings reveal that the proposed RLS model can achieve reduced fluence
reconstruction errors, and potential faster convergence when integrated in an
optimization planner. Additionally, RLS has shown promising results in a full
artificial intelligence RTP pipeline. We hope this pioneer multi-agent RL leaf
sequencer can foster future research on machine learning for RTP.

本文提出了一种新颖的深度强化学习模型 —— 增强叶序列器（RLS），用于叶片排序的多智能体框架，通过大规模训练提供了优化计划中耗时迭代优化步骤的改进，并通过设计奖励机制来控制运动模式。实验结果表明，RLS 模型可以实现减小荧光重建误差，并在优化计划中实现潜在更快的收敛性，同时在全自动智能放疗规划流程中也展示了有希望的结果。希望这种先导性的多智能体强化学习叶片排序器能够促进放射治疗计划中机器学习方面的未来研究。