Federated Learning (FL) enables many resource-limited devices to train a
model collaboratively without data sharing. However, many existing works focus
on model-homogeneous FL, where the global and local models are the same size,
ignoring the inherently heterogeneous computational capabilities of different
devices and restricting resource-constrained devices from contributing to FL.
In this paper, we consider model-heterogeneous FL and propose Federated Partial
Model Training (FedPMT), where devices with smaller computational capabilities
work on partial models (subsets of the global model) and contribute to the
global model. Different from Dropout-based partial model generation, which
removes neurons in hidden layers at random, model training in FedPMT is
achieved from the back-propagation perspective. As such, all devices in FedPMT
prioritize the most crucial parts of the global model. Theoretical analysis
shows that the proposed partial model training design has a similar convergence
rate to the widely adopted Federated Averaging (FedAvg) algorithm,
$\mathcal{O}(1/T)$, with the sub-optimality gap enlarged by a constant factor
related to the model splitting design in FedPMT. Empirical results show that
FedPMT significantly outperforms the existing benchmark FedDrop. Meanwhile,
compared to the popular model-homogeneous benchmark, FedAvg, FedPMT reaches the
learning target in a shorter completion time, thus achieving a better trade-off
between learning accuracy and completion time.

联邦学习通过协作训练模型，避免数据共享，适用于资源有限的设备。本文提出了一种面向模型异构性的联邦学习算法，称为联邦部分模型训练（FedPMT），通过设备之间合作训练局部模型并贡献到全局模型，从而实现全局模型的优化训练。理论分析表明，与传统方法相比，FedPMT 算法具有类似的收敛速度，实证结果也证明其在学习准确性和完成时间之间取得了更好的平衡。

移动边缘网络中基于分层部分模型训练的抗闲杂器弹性联邦学习：应对计算异质性

Straggler-resilient Federated Learning: Tackling Computation  Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network

Federated Learning (FL) is a privacy-constrained decentralized machine
learning paradigm in which clients enable collaborative training without
compromising private data. However, how to learn a robust global model in the
data-heterogeneous and model-heterogeneous FL scenarios is challenging. To
address it, we resort to data-free knowledge distillation to propose a new FL
method (namely DFRD). DFRD equips a conditional generator on the server to
approximate the training space of the local models uploaded by clients, and
systematically investigates its training in terms of fidelity, transferability}
and diversity. To overcome the catastrophic forgetting of the global model
caused by the distribution shifts of the generator across communication rounds,
we maintain an exponential moving average copy of the generator on the server.
Additionally, we propose dynamic weighting and label sampling to accurately
extract knowledge from local models. Finally, our extensive experiments on
various image classification tasks illustrate that DFRD achieves significant
performance gains compared to SOTA baselines.

基于数据异构和模型异构的联邦学习场景中，使用无数据的知识蒸馏机制提出了 DFRD 方法，在服务器上通过条件生成器逼近客户端上传的本地模型训练空间，并通过动态加权和标签采样准确提取本地模型的知识，实验证明 DFRD 相较于基准模型取得了显著的性能提升。