Statistical data heterogeneity is a significant barrier to convergence in federated learning (FL). While prior work has advanced heterogeneous FL through better optimization objectives, these methods fall short when there is extreme data heterogeneity among collaborating participants. We hypothesize that convergence under extreme data heterogeneity is primarily hindered due to the aggregation of conflicting updates from the participants in the initial collaboration rounds. To overcome this problem, we propose a warmup phase where each participant learns a personalized mask and updates only a subnetwork of the full model. This personalized warmup allows the participants to focus initially on learning specific subnetworks tailored to the heterogeneity of their data. After the warmup phase, the participants revert to standard federated optimization, where all parameters are communicated. We empirically demonstrate that the proposed personalized warmup via subnetworks (FedPeWS) approach improves accuracy and convergence speed over standard federated optimization methods.

本研究解决了异构联邦学习中的数据异质性对收敛性造成的显著障碍。我们提出了一种个性化热身阶段，通过学习个性化掩码并只更新全模型的子网络，从而帮助参与者专注于适应数据异质性的特定子网络。实证结果表明，该方法在准确性和收敛速度上均优于标准的联邦优化方法。

通过子网络实现个性化热身以增强异构联邦学习