Federated Learning (FL) enables many resource-limited devices to train a
model collaboratively without data sharing. However, many existing works focus
on model-homogeneous FL, where the global and local models are the same size,
ignoring the inherently heterogeneous computational capabilities of different
devices and restricting resource-constrained devices from contributing to FL.
In this paper, we consider model-heterogeneous FL and propose Federated Partial
Model Training (FedPMT), where devices with smaller computational capabilities
work on partial models (subsets of the global model) and contribute to the
global model. Different from Dropout-based partial model generation, which
removes neurons in hidden layers at random, model training in FedPMT is
achieved from the back-propagation perspective. As such, all devices in FedPMT
prioritize the most crucial parts of the global model. Theoretical analysis
shows that the proposed partial model training design has a similar convergence
rate to the widely adopted Federated Averaging (FedAvg) algorithm,
$\mathcal{O}(1/T)$, with the sub-optimality gap enlarged by a constant factor
related to the model splitting design in FedPMT. Empirical results show that
FedPMT significantly outperforms the existing benchmark FedDrop. Meanwhile,
compared to the popular model-homogeneous benchmark, FedAvg, FedPMT reaches the
learning target in a shorter completion time, thus achieving a better trade-off
between learning accuracy and completion time.

联邦学习通过协作训练模型，避免数据共享，适用于资源有限的设备。本文提出了一种面向模型异构性的联邦学习算法，称为联邦部分模型训练（FedPMT），通过设备之间合作训练局部模型并贡献到全局模型，从而实现全局模型的优化训练。理论分析表明，与传统方法相比，FedPMT 算法具有类似的收敛速度，实证结果也证明其在学习准确性和完成时间之间取得了更好的平衡。