Navigating safely and efficiently in dense and heterogeneous traffic scenarios is challenging for autonomous vehicles (AVs) due to their inability to infer the behaviors or intentions of nearby drivers. In this work, we propose a distributed multi-agent reinforcement learning (MARL) algorithm with trajectory and intent prediction in dense and heterogeneous traffic scenarios. Our approach for intent-aware planning, iPLAN, allows agents to infer nearby drivers' intents solely from their local observations. We model two distinct incentives for agents' strategies: Behavioral incentives for agents' long-term planning based on their driving behavior or personality; Instant incentives for agents' short-term planning for collision avoidance based on the current traffic state. We design a two-stream inference module that allows agents to infer their opponents' incentives and incorporate their inferred information into decision-making. We perform experiments on two simulation environments, Non-Cooperative Navigation and Heterogeneous Highway. In Heterogeneous Highway, results show that, compared with centralized MARL baselines such as QMIX and MAPPO, our method yields a 4.0% and 35.7% higher episodic reward in mild and chaotic traffic, with 48.1% higher success rate and 80.6% longer survival time in chaotic traffic. We also compare with a decentralized baseline IPPO and demonstrate a higher episodic reward of 9.2% and 10.3% in mild traffic and chaotic traffic, 25.3% higher success rate, and 13.7% longer survival time.

本文提出了一种分布式多智能体强化学习算法，采用轨迹和意图预测，以应对自动驾驶汽车在密集交通情况下的安全和高效导航问题，并采用 iPLAN 方法进行意图感知规划，通过设计两种激励模块，促进从行为和长期的驾驶策略与当前交通情况的短期策略方面进行决策，仿真实验结果表明，与集中式 MARL 基线相比，本文提出的方法在混乱交通中的奖励、成功率和幸存时间上均有所提高。

iPLAN: 分布式多智能体强化学习在异质交通中的意图感知规划