Most existing federated learning algorithms are based on the vanilla FedAvg
scheme. However, with the increase of data complexity and the number of model
parameters, the amount of communication traffic and the number of iteration
rounds for training such algorithms increases significantly, especially in
non-independently and homogeneously distributed scenarios, where they do not
achieve satisfactory performance. In this work, we propose FedND: federated
learning with noise distillation. The main idea is to use knowledge
distillation to optimize the model training process. In the client, we propose
a self-distillation method to train the local model. In the server, we generate
noisy samples for each client and use them to distill other clients. Finally,
the global model is obtained by the aggregation of local models. Experimental
results show that the algorithm achieves the best performance and is more
communication-efficient than state-of-the-art methods.

本文提出一种名为 FedND 的新型联邦学习算法，采用知识蒸馏优化模型训练过程，并在客户端使用自我蒸馏方法进行本地模型训练，在服务器端生成噪声样本用于蒸馏其他客户端，最终通过聚合本地模型获得全局模型，实验结果表明该算法不仅达到最佳性能，而且比现有的算法更具通信效率。