Multi-agent reinforcement learning for incomplete information environments
has attracted extensive attention from researchers. However, due to the slow
sample collection and poor sample exploration, there are still some problems in
multi-agent reinforcement learning, such as unstable model iteration and low
training efficiency. Moreover, most of the existing distributed framework are
proposed for single-agent reinforcement learning and not suitable for
multi-agent. In this paper, we design an distributed MARL framework based on
the actor-work-learner architecture. In this framework, multiple asynchronous
environment interaction modules can be deployed simultaneously, which greatly
improves the sample collection speed and sample diversity. Meanwhile, to make
full use of computing resources, we decouple the model iteration from
environment interaction, and thus accelerate the policy iteration. Finally, we
verified the effectiveness of propose framework in MaCA military simulation
environment and the SMAC 3D realtime strategy gaming environment with
imcomplete information characteristics.

本研究提出了一种基于 actor-work-learner 架构的分布式 MARL 框架，在 MaCA 军事模拟环境和 SMAC 3D 实时战略游戏环境中验证了其在多智能体强化学习性能提升方面的有效性。