There is a growing interest in the distributed optimization framework that
goes under the name of Federated Learning (FL). In particular, much attention
is being turned to FL scenarios where the network is strongly heterogeneous in
terms of communication resources (e.g., bandwidth) and data distribution. In
these cases, communication between local machines (agents) and the central
server (Master) is a main consideration. In this work, we present SHED, an
original communication-constrained Newton-type (NT) algorithm designed to
accelerate FL in such heterogeneous scenarios. SHED is by design robust to non
i.i.d. data distributions, handles heterogeneity of agents' communication
resources (CRs), only requires sporadic Hessian computations, and achieves
super-linear convergence. This is possible thanks to an incremental strategy,
based on eigendecomposition of the local Hessian matrices, which exploits
(possibly) outdated second-order information. The proposed solution is
thoroughly validated on real datasets by assessing (i) the number of
communication rounds required for convergence, (ii) the overall amount of data
transmitted and (iii) the number of local Hessian computations. For all these
metrics, the proposed approach shows superior performance against state-of-the
art techniques like GIANT and FedNL.

本研究提出了 SHED 算法，这是一种专门用于加速异构场景中的联邦学习的通信受限牛顿型算法，具有对非独立同分布数据分布的鲁棒性，处理了代理通信资源的异构性，并实现了超线性收敛。