Multi-vehicle pursuit (MVP) such as autonomous police vehicles pursuing
suspects is important but very challenging due to its mission and safety
critical nature. While multi-agent reinforcement learning (MARL) algorithms
have been proposed for MVP problem in structured grid-pattern roads, the
existing algorithms use randomly training samples in centralized learning,
which leads to homogeneous agents showing low collaboration performance. For
the more challenging problem of pursuing multiple evading vehicles, these
algorithms typically select a fixed target evading vehicle for pursuing
vehicles without considering dynamic traffic situation, which significantly
reduces pursuing success rate. To address the above problems, this paper
proposes a Progression Cognition Reinforcement Learning with Prioritized
Experience for MVP (PEPCRL-MVP) in urban multi-intersection dynamic traffic
scenes. PEPCRL-MVP uses a prioritization network to assess the transitions in
the global experience replay buffer according to the parameters of each MARL
agent. With the personalized and prioritized experience set selected via the
prioritization network, diversity is introduced to the learning process of
MARL, which can improve collaboration and task related performance.
Furthermore, PEPCRL-MVP employs an attention module to extract critical
features from complex urban traffic environments. These features are used to
develop progression cognition method to adaptively group pursuing vehicles.
Each group efficiently target one evading vehicle in dynamic driving
environments. Extensive experiments conducted with a simulator over
unstructured roads of an urban area show that PEPCRL-MVP is superior to other
state-of-the-art methods. Specifically, PEPCRL-MVP improves pursuing efficiency
by 3.95% over TD3-DMAP and its success rate is 34.78% higher than that of
MADDPG. Codes are open sourced.

本文提出了一种基于优先体验的进化认知强化学习算法，用于处理多车追逐问题。该算法利用优先级网络对各个智能体的参数进行评估和个性化学习，引入了多样性，以提高协作和任务相关性能。同时，该算法还采用了注意力机制来从复杂的城市交通环境中提取关键特征，并使用进化认知方法自适应地对追逐车辆进行有效分组，进一步提高了追逐效率。

具有优先经验的多车追击的进化认知强化学习

Progression Cognition Reinforcement Learning with Prioritized Experience  for Multi-Vehicle Pursuit

The multi-vehicle pursuit (MVP), as a problem abstracted from various
real-world scenarios, is becoming a hot research topic in Intelligent
Transportation System (ITS). The combination of Artificial Intelligence (AI)
and connected vehicles has greatly promoted the research development of MVP.
However, existing works on MVP pay little attention to the importance of
information exchange and cooperation among pursuing vehicles under the complex
urban traffic environment. This paper proposed a graded-Q reinforcement
learning with information-enhanced state encoder (GQRL-IESE) framework to
address this hierarchical collaborative multi-vehicle pursuit (HCMVP) problem.
In the GQRL-IESE, a cooperative graded Q scheme is proposed to facilitate the
decision-making of pursuing vehicles to improve pursuing efficiency. Each
pursuing vehicle further uses a deep Q network (DQN) to make decisions based on
its encoded state. A coordinated Q optimizing network adjusts the individual
decisions based on the current environment traffic information to obtain the
global optimal action set. In addition, an information-enhanced state encoder
is designed to extract critical information from multiple perspectives and uses
the attention mechanism to assist each pursuing vehicle in effectively
determining the target. Extensive experimental results based on SUMO indicate
that the total timestep of the proposed GQRL-IESE is less than other methods on
average by 47.64%, which demonstrates the excellent pursuing efficiency of the
GQRL-IESE. Codes are outsourced in this https URL.

本文提出了一种基于分级协同增强的强化学习框架（GQRL-IESE）来解决复杂的城市交通环境下的分层多车追逐问题，实验证明该框架的效率比其他方法平均快了 47.64％以上。