In Federated Learning (FL), a framework to train machine learning models
across distributed data, well-known algorithms like FedAvg tend to have slow
convergence rates, resulting in high communication costs during training. To
address this challenge, we introduce FedLion, an adaptive federated
optimization algorithm that seamlessly incorporates key elements from the
recently proposed centralized adaptive algorithm, Lion (Chen et al. 2o23), into
the FL framework. Through comprehensive evaluations on two widely adopted FL
benchmarks, we demonstrate that FedLion outperforms previous state-of-the-art
adaptive algorithms, including FAFED (Wu et al. 2023) and FedDA. Moreover,
thanks to the use of signed gradients in local training, FedLion substantially
reduces data transmission requirements during uplink communication when
compared to existing adaptive algorithms, further reducing communication costs.
Last but not least, this work also includes a novel theoretical analysis,
showcasing that FedLion attains faster convergence rate than established FL
algorithms like FedAvg.

FedLion 是一种自适应联邦优化算法，将 Lion 算法的关键元素无缝地应用于分布式数据训练的 Federated Learning 框架中，通过综合评估表明 FedLion 在自适应算法中胜过现有的最先进算法，并通过使用有符号梯度在本地训练中来减少数据传输要求，进一步降低通信成本，并展示了 FedLion 相对于 FedAvg 等已建立的 FL 算法具有更快的收敛速度。