Traffic light control is important for reducing congestion in urban mobility
systems. This paper proposes a real-time traffic light control method using
deep Q learning. Our approach incorporates a reward function considering queue
lengths, delays, travel time, and throughput. The model dynamically decides
phase changes based on current traffic conditions. The training of the deep Q
network involves an offline stage from pre-generated data with fixed schedules
and an online stage using real-time traffic data. A deep Q network structure
with a "phase gate" component is used to simplify the model's learning task
under different phases. A "memory palace" mechanism is used to address sample
imbalance during the training process. We validate our approach using both
synthetic and real-world traffic flow data on a road intersecting in Hangzhou,
China. Results demonstrate significant performance improvements of the proposed
method in reducing vehicle waiting time (57.1% to 100%), queue lengths (40.9%
to 100%), and total travel time (16.8% to 68.0%) compared to traditional fixed
signal plans.

本研究提出一种实时交通信号灯控制方法，使用深度 Q 学习，并结合奖励函数，考虑排队长度、延迟、旅行时间和吞吐量，通过根据当前交通状况动态决定相位变化。通过使用合成和实际交通流数据在中国杭州的一个路口进行验证，结果表明与传统的固定信号计划相比，该方法显著改善了车辆等待时间（57.1% 至 100%）、排队长度（40.9% 至 100%）和总行程时间（16.8% 至 68.0%）。

强化学习的红绿灯控制

Traffic Light Control with Reinforcement Learning

Adaptive traffic signal control, which adjusts traffic signal timing
according to real-time traffic, has been shown to be an effective method to
reduce traffic congestion. Available works on adaptive traffic signal control
make responsive traffic signal control decisions based on human-crafted
features (e.g. vehicle queue length). However, human-crafted features are
abstractions of raw traffic data (e.g., position and speed of vehicles), which
ignore some useful traffic information and lead to suboptimal traffic signal
controls. In this paper, we propose a deep reinforcement learning algorithm
that automatically extracts all useful features (machine-crafted features) from
raw real-time traffic data and learns the optimal policy for adaptive traffic
signal control. To improve algorithm stability, we adopt experience replay and
target network mechanisms. Simulation results show that our algorithm reduces
vehicle delay by up to 47% and 86% when compared to another two popular traffic
signal control algorithms, longest queue first algorithm and fixed time control
algorithm, respectively.

本文提出了一种利用深度强化学习算法从实时交通原始数据中自动提取所有有用特征（机器制造的特征）并学习适应性交通信号控制的最优策略，以减少车辆延迟时间的方法，并通过模拟实验结果证明，与其他两种流行的交通信号控制算法相比，我们的算法将车辆延迟缩短了 47％和 86％。