Federated learning has emerged as a popular technique for distributing model
training across the network edge. Its learning architecture is conventionally a
star topology between the devices and a central server. In this paper, we
propose two timescale hybrid federated learning (TT-HF), which migrates to a
more distributed topology via device-to-device (D2D) communications. In TT-HF,
local model training occurs at devices via successive gradient iterations, and
the synchronization process occurs at two timescales: (i) macro-scale, where
global aggregations are carried out via device-server interactions, and (ii)
micro-scale, where local aggregations are carried out via D2D cooperative
consensus formation in different device clusters. Our theoretical analysis
reveals how device, cluster, and network-level parameters affect the
convergence of TT-HF, and leads to a set of conditions under which a
convergence rate of O(1/t) is guaranteed. Experimental results demonstrate the
improvements in convergence and utilization that can be obtained by TT-HF over
state-of-the-art federated learning baselines.

本文提出两时间尺度的混合联邦学习，通过设备到设备通信，从星型拓扑迁移到更分布式的拓扑结构。使用两个时间尺度进行同步，进行全局聚合和本地聚合，通过理论分析得出 O（1 /t）收敛率的条件。实验结果表明，TT-HF 在联邦学习基线的收敛和利用方面的改进。

联邦学习超越明星：基于全局集群采样的本地 D2D 模型一致性

Federated Learning Beyond the Star: Local D2D Model Consensus with  Global Cluster Sampling

Federated learning (FL) is a new machine learning framework which trains a
joint model across a large amount of decentralized computing devices. Existing
methods, e.g., Federated Averaging (FedAvg), are able to provide an
optimization guarantee by synchronously training the joint model, but usually
suffer from stragglers, i.e., IoT devices with low computing power or
communication bandwidth, especially on heterogeneous optimization problems. To
mitigate the influence of stragglers, this paper presents a novel FL algorithm,
namely Hybrid Federated Learning (HFL), to achieve a learning balance in
efficiency and effectiveness. It consists of two major components: synchronous
kernel and asynchronous updater. Unlike traditional synchronous FL methods, our
HFL introduces the asynchronous updater which actively pulls unsynchronized and
delayed local weights from stragglers. An adaptive approximation method,
Adaptive Delayed-SGD (AD-SGD), is proposed to merge the delayed local updates
into the joint model. The theoretical analysis of HFL shows that the
convergence rate of the proposed algorithm is $\mathcal{O}(\frac{1}{t+\tau})$
for both convex and non-convex optimization problems.

该研究提出了一种混合联邦学习算法（HFL），通过引入异步更新器和自适应延迟 SGD（AD-SGD）等方法，有效解决了异构优化问题中慢设备所带来的训练效率和学习效果不平衡的问题，且算法收敛速度为 $O (1/t+τ)$。