In cooperative multi-agent reinforcement learning (MARL), the environmental stochasticity and uncertainties will increase exponentially when the number of agents increases, which puts hard pressure on how to come up with a compact latent representation from partial observation for boosting value decomposition. To tackle these issues, we propose a simple yet powerful method that alleviates partial observability and efficiently promotes coordination by introducing the UNit-wise attentive State Representation (UNSR). In UNSR, each agent learns a compact and disentangled unit-wise state representation outputted from transformer blocks, and produces its local action-value function. The proposed UNSR is used to boost the value decomposition with a multi-head attention mechanism for producing efficient credit assignment in the mixing network, providing an efficient reasoning path between the individual value function and joint value function. Experimental results demonstrate that our method achieves superior performance and data efficiency compared to solid baselines on the StarCraft II micromanagement challenge. Additional ablation experiments also help identify the key factors contributing to the performance of UNSR.

该研究提出了一种名为UNSR的方法用于协作多智能体加强学习中通过局部观察产生紧凑的潜在表示进行协调和提高价值分解效率，结果表明该方法比基线算法在StarCraft II微观管理挑战中具有更好的性能和数据效率，并且额外的消融实验帮助确定了UNSR性能的关键因素。

利用单元级注意状态表示增强协作多智能体强化学习的价值分解