This study introduces CycLight, a novel cycle-level deep reinforcement learning (RL) approach for network-level adaptive traffic signal control (NATSC) systems. Unlike most traditional RL-based traffic controllers that focus on step-by-step decision making, CycLight adopts a cycle-level strategy, optimizing cycle length and splits simultaneously using Parameterized Deep Q-Networks (PDQN) algorithm. This cycle-level approach effectively reduces the computational burden associated with frequent data communication, meanwhile enhancing the practicality and safety of real-world applications. A decentralized framework is formulated for multi-agent cooperation, while attention mechanism is integrated to accurately assess the impact of the surroundings on the current intersection. CycLight is tested in a large synthetic traffic grid using the microscopic traffic simulation tool, SUMO. Experimental results not only demonstrate the superiority of CycLight over other state-of-the-art approaches but also showcase its robustness against information transmission delays.

本研究引入了CycLight，一种新颖的基于循环层次的深度强化学习方法，用于网络级自适应交通信号控制系统。与大多数传统的基于强化学习的交通控制器不同，CycLight采用了循环层次的策略，使用参数化深度Q网络算法同时优化循环长度和分割。该循环层次方法有效减少了频繁数据通信的计算负担，同时增强了实际应用的实用性和安全性。为多智能体协作制定了分散的框架，同时整合了注意机制以准确评估当前交叉口周围环境的影响。在大型合成交通网格上使用微观交通仿真工具SUMO对CycLight进行了测试。实验结果不仅证明了CycLight相对于最先进方法的卓越性能，还展示了其对信息传输延迟的稳健性。

CycLight：使用循环级策略学习交通信号配合