Federated learning (FL) is a rapidly growing privacy-preserving collaborative
machine learning paradigm. In practical FL applications, local data from each
data silo reflect local usage patterns. Therefore, there exists heterogeneity
of data distributions among data owners (a.k.a. FL clients). If not handled
properly, this can lead to model performance degradation. This challenge has
inspired the research field of heterogeneous federated learning, which
currently remains open. In this paper, we propose a data heterogeneity-robust
FL approach, FedGSP, to address this challenge by leveraging on a novel concept
of dynamic Sequential-to-Parallel (STP) collaborative training. FedGSP assigns
FL clients to homogeneous groups to minimize the overall distribution
divergence among groups, and increases the degree of parallelism by reassigning
more groups in each round. It is also incorporated with a novel Inter-Cluster
Grouping (ICG) algorithm to assist in group assignment, which uses the centroid
equivalence theorem to simplify the NP-hard grouping problem to make it
solvable. Extensive experiments have been conducted on the non-i.i.d. FEMNIST
dataset. The results show that FedGSP improves the accuracy by 3.7% on average
compared with seven state-of-the-art approaches, and reduces the training time
and communication overhead by more than 90%.

本论文提出了一个数据异构鲁棒的联邦学习方法 ——FedGSP，该方法使用动态序列 - 并行协作训练的新概念，通过将 FL 客户端分配到同质组中最小化组与组之间的整体分布差异，增加并行度并通过一种新颖的 Inter-Cluster Grouping (ICG) 算法来辅助分配，该算法利用重心等价定理简化 NP 难问题，实验结果表明 FedGSP 相比七种现有方法平均提高了 3.7％的准确率，并将训练时间和通信成本降低了 90％以上。