We study how to train personalized models for different tasks on
decentralized devices with limited local data. We propose "Structured
Cooperative Learning (SCooL)", in which a cooperation graph across devices is
generated by a graphical model prior to automatically coordinate mutual
learning between devices. By choosing graphical models enforcing different
structures, we can derive a rich class of existing and novel decentralized
learning algorithms via variational inference. In particular, we show three
instantiations of SCooL that adopt Dirac distribution, stochastic block model
(SBM), and attention as the prior generating cooperation graphs. These EM-type
algorithms alternate between updating the cooperation graph and cooperative
learning of local models. They can automatically capture the cross-task
correlations among devices by only monitoring their model updating in order to
optimize the cooperation graph. We evaluate SCooL and compare it with existing
decentralized learning methods on an extensive set of benchmarks, on which
SCooL always achieves the highest accuracy of personalized models and
significantly outperforms other baselines on communication efficiency. Our code
is available at this https URL

本文介绍了一种 Structured Cooperative Learning（SCooL）的方法，用于训练分布在不同设备上的个性化模型，并在不同的 Cooperation Graph 之间进行自动协调，具有很好的效果和通信效率。

基于图模型先验的结构化合作学习

Structured Cooperative Learning with Graphical Model Priors

Multiagent reinforcement learning (MARL) can solve complex cooperative tasks.
However, the efficiency of existing MARL methods relies heavily on well-defined
reward functions. Multiagent tasks with sparse reward feedback are especially
challenging not only because of the credit distribution problem, but also due
to the low probability of obtaining positive reward feedback. In this paper, we
design a graph network called Cooperation Graph (CG). The Cooperation Graph is
the combination of two simple bipartite graphs, namely, the Agent Clustering
subgraph (ACG) and the Cluster Designating subgraph (CDG). Next, based on this
novel graph structure, we propose a Cooperation Graph Multiagent Reinforcement
Learning (CG-MARL) algorithm, which can efficiently deal with the sparse reward
problem in multiagent tasks. In CG-MARL, agents are directly controlled by the
Cooperation Graph. And a policy neural network is trained to manipulate this
Cooperation Graph, guiding agents to achieve cooperation in an implicit way.
This hierarchical feature of CG-MARL provides space for customized
cluster-actions, an extensible interface for introducing fundamental
cooperation knowledge. In experiments, CG-MARL shows state-of-the-art
performance in sparse reward multiagent benchmarks, including the anti-invasion
interception task and the multi-cargo delivery task.

本文提出了一种基于 Cooperation Graph 结构的 Multiagent Reinforcement Learning（CG-MARL）算法，通过设计一个网络结构来有效处理多智能体领域中的稀疏奖励问题，并在实验中展示出全面领先的性能表现。