This paper proposes a Robust Gradient Classification Framework (RGCF) for Byzantine fault tolerance in distributed stochastic gradient descent. The framework consists of a pattern recognition filter which we train to be able to classify individual gradients as Byzantine by using their direction alone. This filter is robust to an arbitrary number of Byzantine workers for convex as well as non-convex optimisation settings, which is a significant improvement on the prior work that is robust to Byzantine faults only when up to 50% of the workers are Byzantine. This solution does not require an estimate of the number of Byzantine workers; its running time is not dependent on the number of workers and can scale up to training instances with a large number of workers without a loss in performance. We validate our solution by training convolutional neural networks on the MNIST dataset in the presence of Byzantine workers.

本文提出了一个Robust Gradient Classification Framework（RGCF）用于在分布式随机梯度下降中实现拜占庭容错。该框架包括一个模式识别过滤器，我们通过仅使用梯度的方向训练它，使其能够将单个梯度分类为拜占庭式的。此解决方案不需要估计拜占庭式工作者的数量;其运行时间不依赖于工作者的数量，并且可以扩展到具有大量工作者的训练实例而不会丧失性能。

拜占庭容错的随机梯度下降鲁棒分类框架