Microscopic traffic simulation plays a crucial role in transportation
engineering by providing insights into individual vehicle behavior and overall
traffic flow. However, creating a realistic simulator that accurately
replicates human driving behaviors in various traffic conditions presents
significant challenges. Traditional simulators relying on heuristic models
often fail to deliver accurate simulations due to the complexity of real-world
traffic environments. Due to the covariate shift issue, existing imitation
learning-based simulators often fail to generate stable long-term simulations.
In this paper, we propose a novel approach called learner-aware supervised
imitation learning to address the covariate shift problem in multi-agent
imitation learning. By leveraging a variational autoencoder simultaneously
modeling the expert and learner state distribution, our approach augments
expert states such that the augmented state is aware of learner state
distribution. Our method, applied to urban traffic simulation, demonstrates
significant improvements over existing state-of-the-art baselines in both
short-term microscopic and long-term macroscopic realism when evaluated on the
real-world dataset pNEUMA.

本文介绍了一种称为学习者感知的监督式模仿学习模型，用于解决多智能体模仿学习中的协变量转移问题，通过利用变分自动编码器同时建模专家和学习者状态分布，在城市交通仿真中应用，展示了在真实数据集 pNEUMA 上在短期微观和长期宏观仿真中相较于现有 baselines 的显著改进。

LASIL: 面向学习者感知的长期微观交通模拟的有监督模仿学习

LASIL: Learner-Aware Supervised Imitation Learning For Long-term  Microscopic Traffic Simulation

The deployment of Autonomous Vehicles (AVs) poses considerable challenges and
unique opportunities for the design and management of future urban road
infrastructure. In light of this disruptive transformation, the Right-Of-Way
(ROW) composition of road space has the potential to be renewed. Design
approaches and intelligent control models have been proposed to address this
problem, but we lack an operational framework that can dynamically generate ROW
plans for AVs and pedestrians in response to real-time demand. Based on
microscopic traffic simulation, this study explores Reinforcement Learning (RL)
methods for evolving ROW compositions. We implement a centralised paradigm and
a distributive learning paradigm to separately perform the dynamic control on
several road network configurations. Experimental results indicate that the
algorithms have the potential to improve traffic flow efficiency and allocate
more space for pedestrians. Furthermore, the distributive learning algorithm
outperforms its centralised counterpart regarding computational cost (49.55\%),
benchmark rewards (25.35\%), best cumulative rewards (24.58\%), optimal actions
(13.49\%) and rate of convergence. This novel road management technique could
potentially contribute to the flow-adaptive and active mobility-friendly
streets in the AVs era.

该研究探讨了强化学习方法用于发展自主车辆和行人共同占用的行车道之后的实时调控，分析表明分布式强化学习算法比集中式算法在计算成本、奖励基准、最佳累计奖励、最优动作和收敛速度方面具有更好的表现，并且这项新的道路管理技术可能有助于在自主车辆时代实现流动自适应和积极便携的街道。

利用强化学习改进自主车辆与行人的交互的自适应道路配置

Adaptive Road Configurations for Improved Autonomous Vehicle-Pedestrian Interactions using Reinforcement Learning

Car-following refers to a control process in which the following vehicle (FV)
tries to keep a safe distance between itself and the lead vehicle (LV) by
adjusting its acceleration in response to the actions of the vehicle ahead. The
corresponding car-following models, which describe how one vehicle follows
another vehicle in the traffic flow, form the cornerstone for microscopic
traffic simulation and intelligent vehicle development. One major motivation of
car-following models is to replicate human drivers' longitudinal driving
trajectories. To model the long-term dependency of future actions on historical
driving situations, we developed a long-sequence car-following trajectory
prediction model based on the attention-based Transformer model. The model
follows a general format of encoder-decoder architecture. The encoder takes
historical speed and spacing data as inputs and forms a mixed representation of
historical driving context using multi-head self-attention. The decoder takes
the future LV speed profile as input and outputs the predicted future FV speed
profile in a generative way (instead of an auto-regressive way, avoiding
compounding errors). Through cross-attention between encoder and decoder, the
decoder learns to build a connection between historical driving and future LV
speed, based on which a prediction of future FV speed can be obtained. We train
and test our model with 112,597 real-world car-following events extracted from
the Shanghai Naturalistic Driving Study (SH-NDS). Results show that the model
outperforms the traditional intelligent driver model (IDM), a fully connected
neural network model, and a long short-term memory (LSTM) based model in terms
of long-sequence trajectory prediction accuracy. We also visualized the
self-attention and cross-attention heatmaps to explain how the model derives
its predictions.

该研究提出一种基于关注力转换器模型的长序列跟车轨迹预测模型，通过跨层带关注的自注意机制和交叉注意机制在历史驾驶数据和未来 LV 速度数据之间建立联系，从而可以准确预测车辆的 Future FV Speed Profile，在上海自然驾驶研究中该模型表现出比传统智能驾驶模型、全连接神经网络模型、长短时记忆模型更高的准确性。