In the backdrop of an increasingly pressing need for effective urban and
highway transportation systems, this work explores the synergy between
model-based and learning-based strategies to enhance traffic flow management by
use of an innovative approach to the problem of highway ramp metering control
that embeds Reinforcement Learning techniques within the Model Predictive
Control framework. The control problem is formulated as an RL task by crafting
a suitable stage cost function that is representative of the traffic
conditions, variability in the control action, and violations of a
safety-critical constraint on the maximum number of vehicles in queue. An
MPC-based RL approach, which merges the advantages of the two paradigms in
order to overcome the shortcomings of each framework, is proposed to learn to
efficiently control an on-ramp and to satisfy its constraints despite
uncertainties in the system model and variable demands. Finally, simulations
are performed on a benchmark from the literature consisting of a small-scale
highway network. Results show that, starting from an MPC controller that has an
imprecise model and is poorly tuned, the proposed methodology is able to
effectively learn to improve the control policy such that congestion in the
network is reduced and constraints are satisfied, yielding an improved
performance compared to the initial controller.

基于模型和学习策略的协同作用，以及嵌入强化学习技术在模型预测控制框架内，以提高交通流量管理的创新方法为控制问题的公式化，该方法考虑了道路匝道入口流量控制的变数模型和需求不确定性，并通过模拟实验验证了其有效性。

基于模型预测控制的公路匝道计量的强化学习

Reinforcement Learning with Model Predictive Control for Highway Ramp  Metering

As one of the important tools for spatial feature extraction, graph
convolution has been applied in a wide range of fields such as traffic flow
prediction. However, current popular works of graph convolution cannot
guarantee spatio-temporal consistency in a long period. The ignorance of
correlational dynamics, convolutional locality and temporal comprehensiveness
would limit predictive accuracy. In this paper, a novel Attention-based Dynamic
Graph Convolutional Recurrent Neural Network (ADGCRNN) is proposed to improve
traffic flow prediction in highway transportation. Three temporal resolutions
of data sequence are effectively integrated by self-attention to extract
characteristics; multi-dynamic graphs and their weights are dynamically created
to compliantly combine the varying characteristics; a dedicated gated kernel
emphasizing highly relative nodes is introduced on these complete graphs to
reduce overfitting for graph convolution operations. Experiments on two public
datasets show our work better than state-of-the-art baselines, and case studies
of a real Web system prove practical benefit in highway transportation.

提出了一种名为 ADGCRNN 的注意力机制动态图卷积循环神经网络，用于改善高速公路交通中的交通流量预测，通过注意力机制有效地整合了三种时间分辨率的数据序列，动态创建多动态图以结合动态特征，引入专注于高相关节点的门控核以减少图卷积操作的过拟合，实验证明该方法优于现有技术基线。