Traffic signal control (TSC) is a promising low-cost measure to enhance
transportation efficiency without affecting existing road infrastructure. While
various reinforcement learning-based TSC methods have been proposed and
experimentally outperform conventional rule-based methods, none of them has
been deployed in the real world. An essential gap lies in the
oversimplification of the scenarios in terms of intersection heterogeneity and
road network intricacy. To make TSC applicable in urban traffic management, we
target TSC coordination in city-scale high-authenticity road networks, aiming
to solve the three unique and important challenges: city-level scalability,
heterogeneity of real-world intersections, and effective coordination among
intricate neighbor connections. Since optimizing multiple agents in a
parameter-sharing paradigm can boost the training efficiency and help achieve
scalability, we propose our method, CityLight, based on the well-acknowledged
optimization framework, parameter-sharing MAPPO. To ensure the unified policy
network can learn to fit large-scale heterogeneous intersections and tackle the
intricate between-neighbor coordination, CityLight proposes a universal
representation module that consists of two key designs: heterogeneous
intersection alignment and neighborhood impact alignment for coordination. To
further boost coordination, CityLight adopts neighborhood-integrated rewards to
transition from achieving local optimal to global optimal. Extensive
experiments on datasets with hundreds to tens of thousands of real-world
intersections and authentic traffic demands validate the surprising
effectiveness and generalizability of CityLight, with an overall performance
gain of 11.66% and a 22.59% improvement in transfer scenarios in terms of
throughput.

交通信号控制是一项具有潜力的低成本措施，可以提高交通效率而不影响现有道路基础设施。本篇论文提出了基于增强学习的城市规模高可靠道路网络交通信号控制方法 CityLight，通过参数共享的 MAPPO 优化框架来实现多个智能体的协同控制和大规模异质交叉口的适配，采用邻域集成奖励以达到全局最优。大量实验证明 CityLight 具有出人意料的效果和推广性，整体性能提升 11.66%，在迁移场景中吞吐量提高了 22.59%。