Cross-device Federated Learning (FL) faces significant challenges where
low-end clients that could potentially make unique contributions are excluded
from training large models due to their resource bottlenecks. Recent research
efforts have focused on model-heterogeneous FL, by extracting reduced-size
models from the global model and applying them to local clients accordingly.
Despite the empirical success, general theoretical guarantees of convergence on
this method remain an open question. In this paper, we present a unifying
framework for heterogeneous FL algorithms with online model extraction and
provide a general convergence analysis. In particular, we prove that under
certain sufficient conditions and for both IID and non-IID data, these
algorithms converge to a stationary point of standard FL for general smooth
cost functions. Moreover, we illuminate two key factors impacting its
convergence: model-extraction noise and minimum coverage index, advocating a
joint design of local model extraction for efficient heterogeneous FL.

通过在线模型提取，本文提出了一个统一的异构联邦学习算法框架，并进行了一般性的收敛性分析，证明了在某些充分条件下，这些算法对于标准异构联邦学习的一般光滑成本函数会收敛于一个静止点。此外，还阐明了影响收敛的两个关键因素：模型提取噪音和最小覆盖指数，强调了局部模型提取的联合设计在高效异构联邦学习中的重要性。