Cooperative Adaptive Cruise Control (CACC) plays a pivotal role in enhancing traffic efficiency and safety in Connected and Autonomous Vehicles (CAVs). Reinforcement Learning (RL) has proven effective in optimizing complex decision-making processes in CACC, leading to improved system performance and adaptability. Among RL approaches, Multi-Agent Reinforcement Learning (MARL) has shown remarkable potential by enabling coordinated actions among multiple CAVs through Centralized Training with Decentralized Execution (CTDE). However, MARL often faces scalability issues, particularly when CACC vehicles suddenly join or leave the platoon, resulting in performance degradation. To address these challenges, we propose Communication-Aware Reinforcement Learning (CA-RL). CA-RL includes a communication-aware module that extracts and compresses vehicle communication information through forward and backward information transmission modules. This enables efficient cyclic information propagation within the CACC traffic flow, ensuring policy consistency and mitigating the scalability problems of MARL in CACC. Experimental results demonstrate that CA-RL significantly outperforms baseline methods in various traffic scenarios, achieving superior scalability, robustness, and overall system performance while maintaining reliable performance despite changes in the number of participating vehicles.

合作自适应巡航控制（CACC）在连接和自动驾驶汽车（CAVs）中提高交通效率和安全方面扮演了重要角色。强化学习（RL）在优化CACC的复杂决策过程中表现出了有效性，从而改善了系统性能和适应性。多智能体强化学习（MARL）通过集中训练和分布执行（CTDE）使多个CAVs之间实现协调行动，具有显著的潜力。然而，MARL在面临可扩展性问题时往往面临挑战，尤其是CACC车辆突然加入或离开车队时导致性能下降。为了解决这些问题，我们提出了一种通信感知强化学习（CA-RL），包括一个通信感知模块，通过前向和后向信息传输模块提取和压缩车辆通信信息。这样可以在CACC流量中实现高效循环信息传播，确保策略的一致性，并减轻CACC中MARL的可扩展性问题。实验证明，CA-RL在各种交通场景中明显优于基准方法，实现了更好的可扩展性、稳健性和整体系统性能，并能在参与车辆数量变化时保持可靠的性能。

基于通信的协同自适应巡航控制的强化学习