In large-scale multi-agent systems, the large number of agents and complex
game relationship cause great difficulty for policy learning. Therefore,
simplifying the learning process is an important research issue. In many
multi-agent systems, the interactions between agents often happen locally,
which means that agents neither need to coordinate with all other agents nor
need to coordinate with others all the time. Traditional methods attempt to use
pre-defined rules to capture the interaction relationship between agents.
However, the methods cannot be directly used in a large-scale environment due
to the difficulty of transforming the complex interactions between agents into
rules. In this paper, we model the relationship between agents by a complete
graph and propose a novel game abstraction mechanism based on two-stage
attention network (G2ANet), which can indicate whether there is an interaction
between two agents and the importance of the interaction. We integrate this
detection mechanism into graph neural network-based multi-agent reinforcement
learning for conducting game abstraction and propose two novel learning
algorithms GA-Comm and GA-AC. We conduct experiments in Traffic Junction and
Predator-Prey. The results indicate that the proposed methods can simplify the
learning process and meanwhile get better asymptotic performance compared with
state-of-the-art algorithms.

该研究针对大规模多智能体系统，建立一种基于完全图的游戏抽象机制，以简化学习过程，并通过交通路口和猎物 - 捕食者两种实验验证其在状态空间和收敛性能上的优势。