Communication-constrained algorithms for decentralized learning and
optimization rely on local updates coupled with the exchange of compressed
signals. In this context, differential quantization is an effective technique
to mitigate the negative impact of compression by leveraging correlations
between successive iterates. In addition, the use of error feedback, which
consists of incorporating the compression error into subsequent steps, is a
powerful mechanism to compensate for the bias caused by the compression. Under
error feedback, performance guarantees in the literature have so far focused on
algorithms employing a fusion center or a special class of contractive
compressors that cannot be implemented with a finite number of bits. In this
work, we propose a new decentralized communication-efficient learning approach
that blends differential quantization with error feedback. The approach is
specifically tailored for decentralized learning problems where agents have
individual risk functions to minimize subject to subspace constraints that
require the minimizers across the network to lie in low-dimensional subspaces.
This constrained formulation includes consensus or single-task optimization as
special cases, and allows for more general task relatedness models such as
multitask smoothness and coupled optimization. We show that, under some general
conditions on the compression noise, and for sufficiently small step-sizes
$\mu$, the resulting communication-efficient strategy is stable both in terms
of mean-square error and average bit rate: by reducing $\mu$, it is possible to
keep the estimation errors small (on the order of $\mu$) without increasing
indefinitely the bit rate as $\mu\rightarrow 0$. The results establish that, in
the small step-size regime and with a finite number of bits, it is possible to
attain the performance achievable in the absence of compression.

分散学习中受通信限制的算法依赖于局部更新以及压缩信号的交换，本文提出了一种新的分散、通信高效的学习方法，结合了差分量化和误差反馈，在满足低维子空间限制的前提下实现个体风险函数的最小化，实验证明在小步长且有限位数的情况下，可以实现与无压缩情况相当的性能。