Distributed learning is an effective approach to accelerate model training
using multiple workers. However, substantial communication delays emerge
between workers and a parameter server due to massive costs associated with
communicating gradients. SignSGD with majority voting (signSGD-MV) is a simple
yet effective optimizer that reduces communication costs through one-bit
quantization, yet the convergence rates considerably decrease as adversarial
workers increase. In this paper, we show that the convergence rate is invariant
as the number of adversarial workers increases, provided that the number of
adversarial workers is smaller than that of benign workers. The key idea
showing this counter-intuitive result is our novel signSGD with federated
defense (signSGD-FD). Unlike the traditional approaches, signSGD-FD exploits
the gradient information sent by adversarial workers with the proper weights,
which are obtained through gradient sign decoding. Experimental results
demonstrate signSGD-FD achieves superior convergence rates over traditional
algorithms in various adversarial attack scenarios.

这篇论文研究了分布式学习方法在模型训练中的应用，针对通信延迟和收敛速度的问题提出了两种优化器：signSGD-MV 和 signSGD-FD，并通过实验结果验证了 signSGD-FD 在不同对抗攻击场景中相比传统算法具有更好的收敛速度。