Federated learning (FL) is a novel approach to machine learning that allows
multiple edge devices to collaboratively train a model without disclosing their
raw data. However, several challenges hinder the practical implementation of
this approach, especially when devices and the server communicate over wireless
channels, as it suffers from communication and computation bottlenecks in this
case. By utilizing a communication-efficient framework, we propose a novel
zero-order (ZO) method with a one-point gradient estimator that harnesses the
nature of the wireless communication channel without requiring the knowledge of
the channel state coefficient. It is the first method that includes the
wireless channel in the learning algorithm itself instead of wasting resources
to analyze it and remove its impact. The two main difficulties of this work are
that in FL, the objective function is usually not convex, which makes the
extension of FL to ZO methods challenging, and that including the impact of
wireless channels requires extra attention. However, we overcome these
difficulties and comprehensively analyze the proposed zero-order federated
learning (ZOFL) framework. We establish its convergence theoretically, and we
prove a convergence rate of $O(\frac{1}{\sqrt[3]{K}})$ in the nonconvex
setting. We further demonstrate the potential of our algorithm with
experimental results, taking into account independent and identically
distributed (IID) and non-IID device data distributions.

利用通信高效的框架，我们提出了一种新颖的、基于零阶方法的联邦学习（ZOFL）框架，通过利用无线通信信道的特性，在学习算法本身中包含无线信道，而无需分析并消除其影响。

渲染有用的无线环境：一种零阶随机联邦学习方法

Rendering Wireless Environments Useful for Gradient Estimators: A  Zero-Order Stochastic Federated Learning Method

Despite the importance of sparsity in many large-scale applications, there
are few methods for distributed optimization of sparsity-inducing objectives.
In this paper, we present a communication-efficient framework for
L1-regularized optimization in the distributed environment. By viewing
classical objectives in a more general primal-dual setting, we develop a new
class of methods that can be efficiently distributed and applied to common
sparsity-inducing models, such as Lasso, sparse logistic regression, and
elastic net-regularized problems. We provide theoretical convergence guarantees
for our framework, and demonstrate its efficiency and flexibility with a
thorough experimental comparison on Amazon EC2. Our proposed framework yields
speedups of up to 50x as compared to current state-of-the-art methods for
distributed L1-regularized optimization.

本篇论文提出一种基于分布式环境中的 L1 正则化优化问题的通信高效框架，通过将经典目标在更一般的原始对偶设定中进行观察，我们开发了一类新方法，能够有效地分布式和应用于常见的稀疏性诱导模型。

L1 正则化分布式优化：一种高效的对偶原始框架

L1-Regularized Distributed Optimization: A Communication-Efficient  Primal-Dual Framework

Communication remains the most significant bottleneck in the performance of
distributed optimization algorithms for large-scale machine learning. In this
paper, we propose a communication-efficient framework, CoCoA, that uses local
computation in a primal-dual setting to dramatically reduce the amount of
necessary communication. We provide a strong convergence rate analysis for this
class of algorithms, as well as experiments on real-world distributed datasets
with implementations in Spark. In our experiments, we find that as compared to
state-of-the-art mini-batch versions of SGD and SDCA algorithms, CoCoA
converges to the same .001-accurate solution quality on average 25x as quickly.

提出了一种称为 CoCoA 的通信高效的框架，该框架采用原始 - 对偶设置进行本地计算，从而显著减少必要的通信，实验结果表明，相比于 SGD 和 SDCA 算法，CoCoA 在平均 25 倍的时间内收敛到相同的.001 精度解决方案质量。