Federated Learning is a distributed machine learning approach which enables
model training without data sharing. In this paper, we propose a new federated
learning algorithm, Federated Averaging with Client-level Momentum (FedCM), to
tackle problems of partial participation and client heterogeneity in real-world
federated learning applications. FedCM aggregates global gradient information
in previous communication rounds and modifies client gradient descent with a
momentum-like term, which can effectively correct the bias and improve the
stability of local SGD. We provide theoretical analysis to highlight the
benefits of FedCM. We also perform extensive empirical studies and demonstrate
that FedCM achieves superior performance in various tasks and is robust to
different levels of client numbers, participation rate and client
heterogeneity.

本文提出了一种名为 FedCM 的新型联邦平均算法，用于解决现实世界联邦学习应用中的部分参与和客户异质性问题。它在以前的通信轮次中聚合全局梯度信息，并修改客户端梯度下降，可以有效地纠正偏差并提高本地 SGD 的稳定性。作者提供了理论分析和广泛的实证研究表明，FedCM 在各种任务中均具有卓越的性能，并且对不同的客户数量、参与率和客户异质性具有鲁棒性。