In cooperative multi-agent systems, agents jointly take actions and receive a team reward instead of individual rewards. In the absence of individual reward signals, credit assignment mechanisms are usually introduced to discriminate the contributions of different agents so as to achieve effective cooperation. Recently, the value decomposition paradigm has been widely adopted to realize credit assignment, and QMIX has become the state-of-the-art solution. In this paper, we revisit QMIX from two aspects. First, we propose a new perspective on credit assignment measurement and empirically show that QMIX suffers limited discriminability on the assignment of credits to agents. Second, we propose a gradient entropy regularization with QMIX to realize a discriminative credit assignment, thereby improving the overall performance. The experiments demonstrate that our approach can comparatively improve learning efficiency and achieve better performance.

本文针对合作多智能体系统中个体信用分配问题，重新审视QMIX算法。提出一种新的信用分配度量方法，并通过实证研究表明QMIX在信用分配度量方面能力有限。引入基于QMIX的梯度熵正则化策略，以提高信用分配能力、改善机器学习性能。实验表明，我们的方法可以提高学习效率，获得更好的性能表现。

重新审视QMIX: 通过梯度熵正则化实现区分式信用分配