Recent Multi-Agent Reinforcement Learning (MARL) literature has been largely
focused on Centralized Training with Decentralized Execution (CTDE) paradigm.
CTDE has been a dominant approach for both cooperative and mixed environments
due to its capability to efficiently train decentralized policies. While in
mixed environments full autonomy of the agents can be a desirable outcome,
cooperative environments allow agents to share information to facilitate
coordination. Approaches that leverage this technique are usually referred as
communication methods, as full autonomy of agents is compromised for better
performance. Although communication approaches have shown impressive results,
they do not fully leverage this additional information during training phase.
In this paper, we propose a new method called MAMBA which utilizes Model-Based
Reinforcement Learning (MBRL) to further leverage centralized training in
cooperative environments. We argue that communication between agents is enough
to sustain a world model for each agent during execution phase while imaginary
rollouts can be used for training, removing the necessity to interact with the
environment. These properties yield sample efficient algorithm that can scale
gracefully with the number of agents. We empirically confirm that MAMBA
achieves good performance while reducing the number of interactions with the
environment up to an orders of magnitude compared to Model-Free
state-of-the-art approaches in challenging domains of SMAC and Flatland.

本文提出了一种名为 MAMBA 的新方法，通过利用基于模型的强化学习（MBRL）进一步利用合作环境中的集中式训练，从而使代理之间的通信足以在执行阶段维持每个代理的世界模型，而虚拟推演可用于培训，从而通过减少与环境的互动次数，以与 Model-Free 的现有方法相比，在 SMAC 和 Flatland 的具有挑战性的领域中实现良好的性能。

可伸缩的多智能体模型基强化学习

Scalable Multi-Agent Model-Based Reinforcement Learning

The dramatic success of deep learning is largely due to the availability of
data. Data samples are often acquired on edge devices, such as smart phones,
vehicles and sensors, and in some cases cannot be shared due to privacy
considerations. Federated learning is an emerging machine learning paradigm for
training models across multiple edge devices holding local datasets, without
explicitly exchanging the data. Learning in a federated manner differs from
conventional centralized machine learning, and poses several core unique
challenges and requirements, which are closely related to classical problems
studied in the areas of signal processing and communications. Consequently,
dedicated schemes derived from these areas are expected to play an important
role in the success of federated learning and the transition of deep learning
from the domain of centralized servers to mobile edge devices. In this article,
we provide a unified systematic framework for federated learning in a manner
that encapsulates and highlights the main challenges that are natural to treat
using signal processing tools. We present a formulation for the federated
learning paradigm from a signal processing perspective, and survey a set of
candidate approaches for tackling its unique challenges. We further provide
guidelines for the design and adaptation of signal processing and communication
methods to facilitate federated learning at large scale.

本文从信号处理的角度提出联邦学习的理论框架，研究了联邦学习的挑战以及针对这些挑战的一些方法，同时提供设计和适应信号处理和通信方法的指南，以促进联邦学习的大规模实现。