Role-based learning is a promising approach to improving the performance of
Multi-Agent Reinforcement Learning (MARL). Nevertheless, without manual
assistance, current role-based methods cannot guarantee stably discovering a
set of roles to effectively decompose a complex task, as they assume either a
predefined role structure or practical experience for selecting
hyperparameters. In this article, we propose a mathematical Structural
Information principles-based Role Discovery method, namely SIRD, and then
present a SIRD optimizing MARL framework, namely SR-MARL, for multi-agent
collaboration. The SIRD transforms role discovery into a hierarchical action
space clustering. Specifically, the SIRD consists of structuralization,
sparsification, and optimization modules, where an optimal encoding tree is
generated to perform abstracting to discover roles. The SIRD is agnostic to
specific MARL algorithms and flexibly integrated with various value function
factorization approaches. Empirical evaluations on the StarCraft II
micromanagement benchmark demonstrate that, compared with state-of-the-art MARL
algorithms, the SR-MARL framework improves the average test win rate by 0.17%,
6.08%, and 3.24%, and reduces the deviation by 16.67%, 30.80%, and 66.30%,
under easy, hard, and super hard scenarios.

本文提出了一种数学结构化信息原则为基础的角色发现方法 ——SIRD，以及一个基于 SIRD 优化的多智能体协作框架 ——SR-MARL，并在 StarCraft II 微观管理基准测试中证明，SR-MARL 框架相对于最先进的 MARL 算法在易、难和超级难的情况下平均测试胜率分别提高了 0.17%、6.08% 和 3.24%，减少了 16.67%、30.80% 和 66.30% 的偏差。