The present paper develops a novel aggregated gradient approach for
distributed machine learning that adaptively compresses the gradient
communication. The key idea is to first quantize the computed gradients, and
then skip less informative quantized gradient communications by reusing
outdated gradients. Quantizing and skipping result in `lazy' worker-server
communications, which justifies the term Lazily Aggregated Quantized gradient
that is henceforth abbreviated as LAQ. Our LAQ can provably attain the same
linear convergence rate as the gradient descent in the strongly convex case,
while effecting major savings in the communication overhead both in transmitted
bits as well as in communication rounds. Empirically, experiments with real
data corroborate a significant communication reduction compared to existing
gradient- and stochastic gradient-based algorithms.

本文提出了一种新的集成梯度方法来进行分布式机器学习，其通过量化计算的梯度避免信息不足的梯度通信并高效减少通信开销。实验证明，与现有的基于梯度或随机梯度的算法相比，该算法可以显著减少通信量和通信轮数。