Federated Learning (FL) is a machine learning approach where nodes
collaboratively train a global model. As more nodes participate in a round of
FL, the effectiveness of individual model updates by nodes also diminishes. In
this study, we increase the effectiveness of client updates by dividing the
network into smaller partitions, or cohorts. We introduce Cohort-Parallel
Federated Learning (CPFL): a novel learning approach where each cohort
independently trains a global model using FL, until convergence, and the
produced models by each cohort are then unified using one-shot Knowledge
Distillation (KD) and a cross-domain, unlabeled dataset. The insight behind
CPFL is that smaller, isolated networks converge quicker than in a one-network
setting where all nodes participate. Through exhaustive experiments involving
realistic traces and non-IID data distributions on the CIFAR-10 and FEMNIST
image classification tasks, we investigate the balance between the number of
cohorts, model accuracy, training time, and compute and communication
resources. Compared to traditional FL, CPFL with four cohorts, non-IID data
distribution, and CIFAR-10 yields a 1.9$\times$ reduction in train time and a
1.3$\times$ reduction in resource usage, with a minimal drop in test accuracy.

通过将网络划分为较小的分区或队列，CPFL 提出了一种新的学习方法，其中每个队列独立地使用 FL 来培训全局模型，直到收敛，然后使用一次性的知识蒸馏和跨领域、无标签的数据集将各队列生成的模型统一起来。它通过在 CIFAR-10 数据集上进行实际追踪和非 IID 数据分布的详细实验，研究了队列数量、模型准确性、训练时间、计算和通信资源之间的平衡。与传统 FL 相比，CPFL 在拥有四个队列、非 IID 数据分布和 CIFAR-10 的情况下，减少了约 1.9 倍的训练时间和约 1.3 倍的资源使用，只有微小的测试准确率下降。