Spatial information is essential in various fields. How to explicitly model
according to the spatial location of agents is also very important for the
multi-agent problem, especially when the number of agents is changing and the
scale is enormous. Inspired by the point cloud task in computer vision, we
propose a spatial information extraction structure for multi-agent
reinforcement learning in this paper. Agents can effectively share the
neighborhood and global information through a spatially encoder-decoder
structure. Our method follows the centralized training with decentralized
execution (CTDE) paradigm. In addition, our structure can be applied to various
existing mainstream reinforcement learning algorithms with minor modifications
and can deal with the problem with a variable number of agents. The experiments
in several multi-agent scenarios show that the existing methods can get
convincing results by adding our spatially explicit architecture.

本文提出了一种空间信息提取结构，能够针对多智能体强化学习问题中智能体数量变化和规模巨大等问题，通过编码器 - 解码器结构有效地共享邻域和全局信息。该方法遵循中央训练和分散执行（CTDE）模式，并可以应用于各种现有的主流强化学习算法，在细微修改后可处理智能体数量可变的问题。在多个多智能体场景的实验中，通过添加我们的空间显式架构，现有方法都能得到令人信服的结果。

SEA：一种用于多智能体强化学习的空间显式架构

SEA: A Spatially Explicit Architecture for Multi-Agent Reinforcement  Learning

Centralized Training with Decentralized Execution (CTDE) has been a very
popular paradigm for multi-agent reinforcement learning. One of its main
features is making full use of the global information to learn a better joint
$Q$-function or centralized critic. In this paper, we in turn explore how to
leverage the global information to directly learn a better individual
$Q$-function or individual actor. We find that applying the same global
information to all agents indiscriminately is not enough for good performance,
and thus propose to specify the global information for each agent to obtain
agent-specific global information for better performance. Furthermore, we
distill such agent-specific global information into the agent's local
information, which is used during decentralized execution without too much
performance degradation. We call this new paradigm Personalized Training with
Distillated Execution (PTDE). PTDE can be easily combined with many
state-of-the-art algorithms to further improve their performance, which is
verified in both SMAC and Google Research Football scenarios.

本文提出了一种新的多智能体强化学习范式 (PTDE) ，通过对全局信息进行个性化设置，将个体的全局信息提炼成本地信息，用于分权执行，实现更好的性能，其可与现有的算法结合使用，提高性能。