We propose basic and natural assumptions under which iterative optimization
methods with compressed iterates can be analyzed. This problem is motivated by
the practice of federated learning, where a large model stored in the cloud is
compressed before it is sent to a mobile device, which then proceeds with
training based on local data. We develop standard and variance reduced methods,
and establish communication complexity bounds. Our algorithms are the first
distributed methods with compressed iterates, and the first fixed point methods
with compressed iterates.

本文提出了基本而自然的假设，以便分析具有压缩迭代功能的迭代优化方法。我们开发了标准和方差减少的方法，并建立了通信复杂性界限。我们的算法是第一个具有压缩迭代的分布式方法，也是第一个具有压缩迭代的固定点方法。

压缩迭代的分布式不动点方法

Distributed Fixed Point Methods with Compressed Iterates

We propose and analyze a new type of stochastic first order method: gradient
descent with compressed iterates (GDCI). GDCI in each iteration first
compresses the current iterate using a lossy randomized compression technique,
and subsequently takes a gradient step. This method is a distillation of a key
ingredient in the current practice of federated learning, where a model needs
to be compressed by a mobile device before it is sent back to a server for
aggregation. Our analysis provides a step towards closing the gap between the
theory and practice of federated learning, and opens the possibility for many
extensions.

本文提出并分析了一种新型随机一阶方法：使用压缩迭代的梯度下降（GDCI）。GDCI 在每次迭代中首先使用失真随机压缩技术压缩当前迭代，然后进行梯度步骤。该方法是联邦学习实践中的关键要素之一的精华，其需要在将模型发送回服务器进行聚合之前由移动设备压缩。我们的分析为联邦学习的理论与实践之间的差距提供了一步，并开启了许多拓展的可能性。