With more regulations tackling users' privacy-sensitive data protection in
recent years, access to such data has become increasingly restricted and
controversial. To exploit the wealth of data generated and located at
distributed entities such as mobile phones, a revolutionary decentralized
machine learning setting, known as Federated Learning, enables multiple clients
located at different geographical locations to collaboratively learn a machine
learning model while keeping all their data on-device. However, the scale and
decentralization of federated learning present new challenges. Communication
between the clients and the server is considered a main bottleneck in the
convergence time of federated learning.
In this paper, we propose and study Adaptive Federated Dropout (AFD), a novel
technique to reduce the communication costs associated with federated learning.
It optimizes both server-client communications and computation costs by
allowing clients to train locally on a selected subset of the global model. We
empirically show that this strategy, combined with existing compression
methods, collectively provides up to 57x reduction in convergence time. It also
outperforms the state-of-the-art solutions for communication efficiency.
Furthermore, it improves model generalization by up to 1.7%.

本文介绍了一种名为自适应联邦 Dropout（AFD）的新型技术，用于减少联邦学习中与通信成本相关的问题，此策略与现有压缩方法相结合可以提供高达 57 倍的收敛时间，同时还改善了模型泛化。