Federated learning (FL) has emerged as a prominent distributed learning
paradigm. FL entails some pressing needs for developing novel parameter
estimation approaches with theoretical guarantees of convergence, which are
also communication efficient, differentially private and Byzantine resilient in
the heterogeneous data distribution settings. Quantization-based SGD solvers
have been widely adopted in FL and the recently proposed SIGNSGD with majority
vote shows a promising direction. However, no existing methods enjoy all the
aforementioned properties. In this paper, we propose an intuitively-simple yet
theoretically-sound method based on SIGNSGD to bridge the gap. We present
Stochastic-Sign SGD which utilizes novel stochastic-sign based gradient
compressors enabling the aforementioned properties in a unified framework. We
also present an error-feedback variant of the proposed Stochastic-Sign SGD
which further improves the learning performance in FL. We test the proposed
method with extensive experiments using deep neural networks on the MNIST
dataset and the CIFAR-10 dataset. The experimental results corroborate the
effectiveness of the proposed method.

本研究提出一种基于 SIGNSGD 的方法：Stochastic-Sign SGD，它利用新颖的随机符号梯度压缩器，在分布式学习中具有收敛性、通信效率、差分隐私以及拜占庭容错等特性，并在 MNIST 和 CIFAR-10 数据集下的深度神经网络上进行了广泛的实验验证。