Federated learning (FedL) has emerged as a popular technique for distributing
model training over a set of wireless devices, via iterative local updates (at
devices) and global aggregations (at the server). In this paper, we develop
parallel successive learning (PSL), which expands the FedL architecture along
three dimensions: (i) Network, allowing decentralized cooperation among the
devices via device-to-device (D2D) communications. (ii) Heterogeneity,
interpreted at three levels: (ii-a) Learning: PSL considers heterogeneous
number of stochastic gradient descent iterations with different mini-batch
sizes at the devices; (ii-b) Data: PSL presumes a dynamic environment with data
arrival and departure, where the distributions of local datasets evolve over
time, captured via a new metric for model/concept drift. (ii-c) Device: PSL
considers devices with different computation and communication capabilities.
(iii) Proximity, where devices have different distances to each other and the
access point. PSL considers the realistic scenario where global aggregations
are conducted with idle times in-between them for resource efficiency
improvements, and incorporates data dispersion and model dispersion with local
model condensation into FedL. Our analysis sheds light on the notion of cold
vs. warmed up models, and model inertia in distributed machine learning. We
then propose network-aware dynamic model tracking to optimize the model
learning vs. resource efficiency tradeoff, which we show is an NP-hard
signomial programming problem. We finally solve this problem through proposing
a general optimization solver. Our numerical results reveal new findings on the
interdependencies between the idle times in-between the global aggregations,
model/concept drift, and D2D cooperation configuration.

本文对联邦学习进行了拓展，提出了并行连续学习 (PSL)，利用 PSL 的框架，实现了从三个方面的拓展：网络（通过设备之间的设备对设备通信进行分散合作），异构性（考虑到训练设备之间的数量、数据来源和性能差异），以及接近度（设备之间不同的距离和不同的访问点）。PSL 实现了模型的分散和本地模型凝聚作为联邦学习方法的一部分，并提出了网络感知动态模型跟踪以优化模型学习和资源效率之间的权衡。

异构无线网络中动态分布式模型训练的并行连续学习

Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks

Decentralized cooperation in partially-observable multi-agent systems
requires effective communications among agents. To support this effort, this
work focuses on the class of problems where global communications are available
but may be unreliable, thus precluding differentiable communication learning
methods. We introduce FCMNet, a reinforcement learning based approach that
allows agents to simultaneously learn a) an effective multi-hop communications
protocol and b) a common, decentralized policy that enables team-level
decision-making. Specifically, our proposed method utilizes the hidden states
of multiple directional recurrent neural networks as communication messages
among agents. Using a simple multi-hop topology, we endow each agent with the
ability to receive information sequentially encoded by every other agent at
each time step, leading to improved global cooperation. We demonstrate FCMNet
on a challenging set of StarCraft II micromanagement tasks with shared rewards,
as well as a collaborative multi-agent pathfinding task with individual
rewards. There, our comparison results show that FCMNet outperforms
state-of-the-art communication-based reinforcement learning methods in all
StarCraft II micromanagement tasks, and value decomposition methods in certain
tasks. We further investigate the robustness of FCMNet under realistic
communication disturbances, such as random message loss or binarized messages
(i.e., non-differentiable communication channels), to showcase FMCNet's
potential applicability to robotic tasks under a variety of real-world
conditions.

该研究基于强化学习，提出了 FCMNet 通信框架，通过方向性递归神经网络的隐藏状态作为通信信息，在具备全局通信但不可靠的情况下，实现了一个既能学习多跳通信协议，又能让代理团队制定决策的共同分散式智策略。FCMNet 具有较强的鲁棒性，在一系列 StarCraft II 微操作任务中超越了现有的基于通信强化学习方法，也在一些任务中优于价值分解方法，同时还能应对随机丢包或二元信息等通信干扰情况，有望应用于各种实际机器人任务。