In this article, we propose a communication-efficient decentralized machine
learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). To reduce the
number of communication links, every worker in Q-GADMM communicates only with
two neighbors, while updating its model via the group alternating direction
method of multipliers (GADMM). Moreover, each worker transmits the quantized
difference between its current model and its previously quantized model,
thereby decreasing the communication payload size. However, due to the lack of
centralized entity in decentralized ML, the spatial sparsity and payload
compression may incur error propagation, hindering model training convergence.
To overcome this, we develop a novel stochastic quantization method to
adaptively adjust model quantization levels and their probabilities, while
proving the convergence of Q-GADMM for convex objective functions. Furthermore,
to demonstrate the feasibility of Q-GADMM for non-convex and stochastic
problems, we propose quantized stochastic GADMM (Q-SGADMM) that incorporates
deep neural network architectures and stochastic sampling. Simulation results
corroborate that Q-GADMM significantly outperforms GADMM in terms of
communication efficiency while achieving the same accuracy and convergence
speed for a linear regression task. Similarly, for an image classification task
using DNN, Q-SGADMM achieves significantly less total communication cost with
identical accuracy and convergence speed compared to its counterpart without
quantization, i.e., stochastic GADMM (SGADMM).

本文提出了一种通信高效的、去中心化的机器学习算法 —— 量化组 ADMM (Q-GADMM)，通过量化减少通信链接，采用随机量化方法进行误差传播控制，以及利用深度神经网络架构与随机采样的量化随机 GADMM (Q-SGADMM)，通过对多项测试结果的对比表明，该算法在通信效率方面明显优于 GADMM，在同等时间内实现了相同的准确率和收敛速度。