It has long been recognized that multi-agent reinforcement learning (MARL)
faces significant scalability issues due to the fact that the size of the state
and action spaces are exponentially large in the number of agents. In this
paper, we identify a rich class of networked MARL problems where the model
exhibits a local dependence structure that allows it to be solved in a scalable
manner. Specifically, we propose a Scalable Actor-Critic (SAC) method that can
learn a near optimal localized policy for optimizing the average reward with
complexity scaling with the state-action space size of local neighborhoods, as
opposed to the entire network. Our result centers around identifying and
exploiting an exponential decay property that ensures the effect of agents on
each other decays exponentially fast in their graph distance.

本文提出了一种可伸缩的演员 - 评论家（SAC）方法，可以解决具有本地依赖结构的网络多智能体强化学习（MARL）问题，其复杂度与本地邻域的状态 - 动作空间大小相比，而不是整个网络的规模，其效果取决于智能体在图中的距离，通过利用指数衰减性质，可以获得性能接近最优的局部策略。