In this work, we consider the resilience of distributed algorithms based on
stochastic gradient descent (SGD) in distributed learning with potentially
Byzantine attackers, who could send arbitrary information to the parameter
server to disrupt the training process. Toward this end, we propose a new
Lipschitz-inspired coordinate-wise median approach (LICM-SGD) to mitigate
Byzantine attacks. We show that our LICM-SGD algorithm can resist up to half of
the workers being Byzantine attackers, while still converging almost surely to
a stationary region in non-convex settings. Also, our LICM-SGD method does not
require any information about the number of attackers and the Lipschitz
constant, which makes it attractive for practical implementations. Moreover,
our LICM-SGD method enjoys the optimal $O(md)$ computational time-complexity in
the sense that the time-complexity is the same as that of the standard SGD
under no attacks. We conduct extensive experiments to show that our LICM-SGD
algorithm consistently outperforms existing methods in training multi-class
logistic regression and convolutional neural networks with MNIST and CIFAR-10
datasets. In our experiments, LICM-SGD also achieves a much faster running time
thanks to its low computational time-complexity.

本文提出了一种新的利普希茨坐标中值方法 (LICM-SGD) 来降低拜占庭攻击对分布式学习中基于随机梯度下降 (SGD) 的分布式算法的影响。该算法不需要关于攻击者数量和利普希茨常数的任何信息，在实现中具有吸引力，并且在多分类逻辑回归和卷积神经网络的训练中的 MNIST 和 CIFAR-10 数据集上，该算法始终表现出色，并且具有更快的运行时间。