Value decomposition multi-agent reinforcement learning methods learn the
global value function as a mixing of each agent's individual utility functions.
Coordination graphs (CGs) represent a higher-order decomposition by
incorporating pairwise payoff functions and thus is supposed to have a more
powerful representational capacity. However, CGs decompose the global value
function linearly over local value functions, severely limiting the complexity
of the value function class that can be represented. In this paper, we propose
the first non-linear coordination graph by extending CG value decomposition
beyond the linear case. One major challenge is to conduct greedy action
selections in this new function class to which commonly adopted DCOP algorithms
are no longer applicable. We study how to solve this problem when mixing
networks with LeakyReLU activation are used. An enumeration method with a
global optimality guarantee is proposed and motivates an efficient iterative
optimization method with a local optimality guarantee. We find that our method
can achieve superior performance on challenging multi-agent coordination tasks
like MACO.

本研究提出了第一个非线性协调图，探讨如何在新的函数类中进行贪心行动选择，使用 LeakyReLU 激活的混合网络解决此问题，提出具有全局最优性保证的枚举方法和具有局部最优性保证的有效迭代优化方法的动机。在多个代理协调任务中取得了卓越性能。