Adaptive optimization has achieved notable success for distributed learning while extending adaptive optimizer to federated Learning (FL) suffers from severe inefficiency, including (i) rugged convergence due to inaccurate gradient estimation in global adaptive optimizer; (ii) client drifts exacerbated by local over-fitting with the local adaptive optimizer. In this work, we propose a novel momentum-based algorithm via utilizing the global gradient descent and locally adaptive amended optimizer to tackle these difficulties. Specifically, we incorporate a locally amended technique to the adaptive optimizer, named Federated Local ADaptive Amended optimizer (\textit{FedLADA}), which estimates the global average offset in the previous communication round and corrects the local offset through a momentum-like term to further improve the empirical training speed and mitigate the heterogeneous over-fitting. Theoretically, we establish the convergence rate of \textit{FedLADA} with a linear speedup property on the non-convex case under the partial participation settings. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficacy of our proposed \textit{FedLADA}, which could greatly reduce the communication rounds and achieves higher accuracy than several baselines.

通过利用全局梯度下降和本地自适应修正优化器，提出了一种新颖的基于动量的算法来解决分布式学习中的不准确梯度估计以及局部自适应优化器导致的收敛困难和客户端漂移等问题。在理论上，我们在部分参与设置下建立了FedLADA的收敛速度与线性加速特性。此外，我们在真实世界的数据集上进行了广泛的实验，证明了我们所提出的FedLADA的有效性，它可以大大减少通信轮数，并且比几个基准模型达到更高的准确性。

基于本地自适应优化的高效联邦学习与线性加速