The capability to learn and adapt to changes in the driving environment is
crucial for developing autonomous driving systems that are scalable beyond
geo-fenced operational design domains. Deep Reinforcement Learning (RL)
provides a promising and scalable framework for developing adaptive learning
based solutions. Deep RL methods usually model the problem as a (Partially
Observable) Markov Decision Process in which an agent acts in a stationary
environment to learn an optimal behavior policy. However, driving involves
complex interaction between multiple, intelligent (artificial or human) agents
in a highly non-stationary environment. In this paper, we propose the use of
Partially Observable Markov Games(POSG) for formulating the connected
autonomous driving problems with realistic assumptions. We provide a taxonomy
of multi-agent learning environments based on the nature of tasks, nature of
agents and the nature of the environment to help in categorizing various
autonomous driving problems that can be addressed under the proposed
formulation. As our main contributions, we provide MACAD-Gym, a Multi-Agent
Connected, Autonomous Driving agent learning platform for furthering research
in this direction. Our MACAD-Gym platform provides an extensible set of
Connected Autonomous Driving (CAD) simulation environments that enable the
research and development of Deep RL- based integrated sensing, perception,
planning and control algorithms for CAD systems with unlimited operational
design domain under realistic, multi-agent settings. We also share the
MACAD-Agents that were trained successfully using the MACAD-Gym platform to
learn control policies for multiple vehicle agents in a partially observable,
stop-sign controlled, 3-way urban intersection environment with raw (camera)
sensor observations.

该论文提出了使用部分可观测马尔可夫博弈来构建具有现实情况的无人驾驶问题，并提供了 MACAD-Gym 平台来进一步研究和开发基于深度强化学习的集成感知、计划和控制算法。