Autonomous Vehicles (AVs), furnished with sensors capable of capturing
essential vehicle dynamics such as speed, acceleration, and precise location,
possess the capacity to execute intelligent maneuvers, including lane changes,
in anticipation of approaching roadblocks. Nevertheless, the sheer volume of
sensory data and the processing necessary to derive informed decisions can
often overwhelm the vehicles, rendering them unable to handle the task
independently. Consequently, a common approach in traffic scenarios involves
transmitting the data to servers for processing, a practice that introduces
challenges, particularly in situations demanding real-time processing. In
response to this challenge, we present a novel DL-based semantic traffic
control system that entrusts semantic encoding responsibilities to the vehicles
themselves. This system processes driving decisions obtained from a
Reinforcement Learning (RL) agent, streamlining the decision-making process.
Specifically, our framework envisions scenarios where abrupt roadblocks
materialize due to factors such as road maintenance, accidents, or vehicle
repairs, necessitating vehicles to make determinations concerning lane-keeping
or lane-changing actions to navigate past these obstacles. To formulate this
scenario mathematically, we employ a Markov Decision Process (MDP) and harness
the Deep Q Learning (DQN) algorithm to unearth viable solutions.

自动驾驶车辆利用传感器捕捉车辆动力学信息，包括速度、加速度和精确位置，执行智能转向等操作以应对道路障碍。本文提出了一种新颖的基于深度学习的语义交通控制系统，使车辆能够自行处理驾驶决策，从而简化决策过程，并利用强化学习和深度 Q 学习算法来找到可行的解决方案。

基于 RL 和 DQN 的 AV 分布式语义交通控制与动态道路封闭

Decentralized Semantic Traffic Control in AVs Using RL and DQN for  Dynamic Roadblocks

Traffic light control is important for reducing congestion in urban mobility
systems. This paper proposes a real-time traffic light control method using
deep Q learning. Our approach incorporates a reward function considering queue
lengths, delays, travel time, and throughput. The model dynamically decides
phase changes based on current traffic conditions. The training of the deep Q
network involves an offline stage from pre-generated data with fixed schedules
and an online stage using real-time traffic data. A deep Q network structure
with a "phase gate" component is used to simplify the model's learning task
under different phases. A "memory palace" mechanism is used to address sample
imbalance during the training process. We validate our approach using both
synthetic and real-world traffic flow data on a road intersecting in Hangzhou,
China. Results demonstrate significant performance improvements of the proposed
method in reducing vehicle waiting time (57.1% to 100%), queue lengths (40.9%
to 100%), and total travel time (16.8% to 68.0%) compared to traditional fixed
signal plans.

本研究提出一种实时交通信号灯控制方法，使用深度 Q 学习，并结合奖励函数，考虑排队长度、延迟、旅行时间和吞吐量，通过根据当前交通状况动态决定相位变化。通过使用合成和实际交通流数据在中国杭州的一个路口进行验证，结果表明与传统的固定信号计划相比，该方法显著改善了车辆等待时间（57.1% 至 100%）、排队长度（40.9% 至 100%）和总行程时间（16.8% 至 68.0%）。

强化学习的红绿灯控制

Traffic Light Control with Reinforcement Learning

This paper presents a methodology for combining programming and mathematics
to optimize elevator wait times. Based on simulated user data generated
according to the canonical three-peak model of elevator traffic, we first
develop a naive model from an intuitive understanding of the logic behind
elevators. We take into consideration a general array of features including
capacity, acceleration, and maximum wait time thresholds to adequately model
realistic circumstances. Using the same evaluation framework, we proceed to
develop a Deep Q Learning model in an attempt to match the hard-coded naive
approach for elevator control. Throughout the majority of the paper, we work
under a Markov Decision Process (MDP) schema, but later explore how the
assumption fails to characterize the highly stochastic overall Elevator Group
Control System (EGCS).

该论文提出了一种方法来结合编程和数学以优化电梯等待时间，基于模拟的用户数据，开发了一个 naive 模型和 Deep Q Learning 模型，研究使用 Markov 决策过程模拟电梯调度但未能 adequately model highly stochastic overall Elevator Group Control System（EGCS）。