Multi-Agent Reinforcement Learning (MARL) methods find optimal policies for
agents that operate in the presence of other learning agents. Central to
achieving this is how the agents coordinate. One way to coordinate is by
learning to communicate with each other. Can the agents develop a language
while learning to perform a common task? In this paper, we formulate and study
a MARL problem where cooperative agents are connected to each other via a fixed
underlying network. These agents can communicate along the edges of this
network by exchanging discrete symbols. However, the semantics of these symbols
are not predefined and, during training, the agents are required to develop a
language that helps them in accomplishing their goals. We propose a method for
training these agents using emergent communication. We demonstrate the
applicability of the proposed framework by applying it to the problem of
managing traffic controllers, where we achieve state-of-the-art performance as
compared to a number of strong baselines. More importantly, we perform a
detailed analysis of the emergent communication to show, for instance, that the
developed language is grounded and demonstrate its relationship with the
underlying network topology. To the best of our knowledge, this is the only
work that performs an in depth analysis of emergent communication in a
networked MARL setting while being applicable to a broad class of problems.

本研究使用紧密联系的智能体通过互相交流离散符号彼此合作完成任务。通过分析他们之间的交流，证明了他们发展的语言与网络拓扑有关，并在交通控制器问题上实现了最先进的性能。