Increasingly available city data and advanced learning techniques have
empowered people to improve the efficiency of our city functions. Among them,
improving the urban transportation efficiency is one of the most prominent
topics. Recent studies have proposed to use reinforcement learning (RL) for
traffic signal control. Different from traditional transportation approaches
which rely heavily on prior knowledge, RL can learn directly from the feedback.
On the other side, without a careful model design, existing RL methods
typically take a long time to converge and the learned models may not be able
to adapt to new scenarios. For example, a model that is trained well for
morning traffic may not work for the afternoon traffic because the traffic flow
could be reversed, resulting in a very different state representation. In this
paper, we propose a novel design called FRAP, which is based on the intuitive
principle of phase competition in traffic signal control: when two traffic
signals conflict, priority should be given to one with larger traffic movement
(i.e., higher demand). Through the phase competition modeling, our model
achieves invariance to symmetrical cases such as flipping and rotation in
traffic flow. By conducting comprehensive experiments, we demonstrate that our
model finds better solutions than existing RL methods in the complicated
all-phase selection problem, converges much faster during training, and
achieves superior generalizability for different road structures and traffic
conditions.

本文提出了一种基于相位竞争模型的交通信号控制方法 FRAP，它利用强化学习算法实现了自适应，具备更好的解决方案和更快的收敛速度，同时在不同的道路结构和交通情况下实现了卓越的普适性。