Federated learning (FL) is a popular machine learning technique that enables multiple users to collaboratively train a model while maintaining the user data privacy. A significant challenge in FL is the communication bottleneck in the upload direction, and thus the corresponding energy consumption of the devices, attributed to the increasing size of the model/gradient. In this paper, we address this issue by proposing a zero-order (ZO) optimization method that requires the upload of a quantized single scalar per iteration by each device instead of the whole gradient vector. We prove its theoretical convergence and find an upper bound on its convergence rate in the non-convex setting, and we discuss its implementation in practical scenarios. Our FL method and the corresponding convergence analysis take into account the impact of quantization and packet dropping due to wireless errors. We show also the superiority of our method, in terms of communication overhead and energy consumption, as compared to standard gradient-based FL methods.

本研究解决了联邦学习中上传方向的通信瓶颈及能量消耗问题，提出了一种零阶优化方法，允许每个设备每次迭代仅需上传一个量化的单标量，而不是整个梯度向量。我们的实验与理论证明表明，该方法在通信开销和能量消耗方面优于传统的基于梯度的联邦学习方法。

基于零阶优化技术的通信与能量高效的联邦学习