There are two paradigms in Federated Learning (FL): parallel FL (PFL), where
models are trained in a parallel manner across clients; and sequential FL
(SFL), where models are trained in a sequential manner across clients. In
contrast to that of PFL, the convergence theory of SFL on heterogeneous data is
still lacking. To resolve the theoretical dilemma of SFL, we establish sharp
convergence guarantees for SFL on heterogeneous data with both upper and lower
bounds. Specifically, we derive the upper bounds for strongly convex, general
convex and non-convex objective functions, and construct the matching lower
bounds for the strongly convex and general convex objective functions. Then, we
compare the upper bounds of SFL with those of PFL, showing that SFL outperforms
PFL (at least, when the level of heterogeneity is relatively high).
Experimental results on quadratic functions and real data sets validate the
counterintuitive comparison result.

对于异构数据上的序列联邦学习 (SFL) 的收敛理论尚缺乏解决方案，本研究通过建立上下界，针对强凸、一般凸和非凸目标函数分别得出上界，以及匹配的强凸和一般凸目标函数的下界，将 SFL 的上界与平行联邦学习 (PFL) 的上界进行对比，结果表明在高度异构的情况下，SFL 较 PFL 表现更出色，并通过二次函数和真实数据集的实验结果验证了这一反直觉的比较结果。

异构数据上连续联合学习的尖锐界限

Sharp Bounds for Sequential Federated Learning on Heterogeneous Data

There are two categories of methods in Federated Learning (FL) for joint
training across multiple clients: i) parallel FL (PFL), where clients train
models in a parallel manner; and ii) sequential FL (SFL), where clients train
models in a sequential manner. In contrast to that of PFL, the convergence
theory of SFL on heterogeneous data is still lacking. In this paper, we
establish the convergence guarantees of SFL for strongly/general/non-convex
objectives on heterogeneous data. The convergence guarantees of SFL are better
than that of PFL on heterogeneous data with both full and partial client
participation. Experimental results validate the counterintuitive analysis
result that SFL outperforms PFL on extremely heterogeneous data in cross-device
settings.

本文研究了联邦学习（FL）方法中两种类别：并行 FL（PFL）和顺序 FL（SFL）。对于异构数据上的顺序 FL，相比并行 FL，其收敛理论仍然欠缺。本文建立了异构数据上强烈 / 一般 / 非凸目标的顺序 FL 的收敛保证。实验结果验证了在跨设备环境中，对于极度异构数据，SFL 优于 PFL 的非直观分析结果。