Federated learning is a promising distributed training paradigm that effectively safeguards data privacy. However, it may involve significant communication costs, which hinders training efficiency. In this paper, we aim to enhance communication efficiency from a new perspective. Specifically, we request the distributed clients to find optimal model updates relative to global model parameters within predefined random noise. For this purpose, we propose Federated Masked Random Noise (FedMRN), a novel framework that enables clients to learn a 1-bit mask for each model parameter and apply masked random noise (i.e., the Hadamard product of random noise and masks) to represent model updates. To make FedMRN feasible, we propose an advanced mask training strategy, called progressive stochastic masking (PSM). After local training, each client only need to transmit local masks and a random seed to the server. Additionally, we provide theoretical guarantees for the convergence of FedMRN under both strongly convex and non-convex assumptions. Extensive experiments are conducted on four popular datasets. The results show that FedMRN exhibits superior convergence speed and test accuracy compared to relevant baselines, while attaining a similar level of accuracy as FedAvg.

本研究针对联邦学习中存在的高通信成本问题，提出了一种新的方法——联邦掩码随机噪声（FedMRN）框架。该框架能够通过学习每个模型参数的1位掩码，并应用掩码随机噪声来表示模型更新，从而提高通信效率。实验结果表明，FedMRN在收敛速度和测试准确率上优于相关基准，同时与FedAvg达到相似的准确性。

基于掩码随机噪声的通信高效联邦学习