Highway merging scenarios featuring mixed traffic conditions pose significant
modeling and control challenges for connected and automated vehicles (CAVs)
interacting with incoming on-ramp human-driven vehicles (HDVs). In this paper,
we present an approach to learn an approximate information state model of
CAV-HDV interactions for a CAV to maneuver safely during highway merging. In
our approach, the CAV learns the behavior of an incoming HDV using approximate
information states before generating a control strategy to facilitate merging.
First, we validate the efficacy of this framework on real-world data by using
it to predict the behavior of an HDV in mixed traffic situations extracted from
the Next-Generation Simulation repository. Then, we generate simulation data
for HDV-CAV interactions in a highway merging scenario using a standard inverse
reinforcement learning approach. Without assuming a prior knowledge of the
generating model, we show that our approximate information state model learns
to predict the future trajectory of the HDV using only observations.
Subsequently, we generate safe control policies for a CAV while merging with
HDVs, demonstrating a spectrum of driving behaviors, from aggressive to
conservative. We demonstrate the effectiveness of the proposed approach by
performing numerical simulations.

该论文提出了一种学习连接和自动化车辆 (CAV) 与进入匝道的人驾驶车辆 (HDV) 进行安全控制的方法，使用高度信息状态模型，学习了混合交通情况下 HDV 的行为，并生成安全控制策略，然后在 CAV 与 HDV 交互的模拟场景中进行了验证。

混合交通中的联网自动驾驶车辆：学习人类驾驶员行为以实现有效的匝道并入

Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver  Behavior for Effective On-Ramp Merging

This paper focuses on a drag-reducing control strategy on a 2D-simulated
laminar flow past a cylinder. Deep reinforcement learning algorithms have been
implemented to discover efficient control schemes, using two synthetic jets
located on the cylinder's poles as actuators and pressure sensors in the wake
of the cylinder as feedback observation. The present work focuses on the
efficiency and robustness of the identified control strategy and introduces a
novel algorithm (S-PPO-CMA) to optimise the sensor layout. An energy-efficient
control strategy reducing drag by 18.4% at Reynolds number 120 is obtained.
This control policy is shown to be robust both to the Reynolds number in the
range [100,216] and to measurement noise, enduring signal to noise ratios as
low as 0.2 with negligible impact on performance. Along with a systematic study
on sensor number and location, the proposed sparsity-seeking algorithm has
achieved a successful optimisation to a reduced 5-sensor layout while keeping
state-of-the-art performance. These results highlight the interesting
possibilities of reinforcement learning for active flow control and pave the
way to efficient, robust and practical implementations of these control
techniques in experimental or industrial systems.

本研究使用深度强化学习算法，结合两个人工喷射器和压力传感器在圆柱的尾迹中，探索减少阻力的控制策略，并提出了新的优化算法（S-PPO-CMA）来优化传感器布局，结果显示控制效果稳定、鲁棒性高。这些成功实现的结果证明了强化学习在主动流控中具有非常大的潜力，并为实验和工业系统的高效、鲁棒和实用的控制技术实现铺平了道路。