This report considers the problem of Byzantine fault-tolerance in synchronous
parallelized learning that is founded on the parallelized stochastic gradient
descent (parallelized-SGD) algorithm. The system comprises a master, and $n$
workers, where up to $f$ of the workers are Byzantine faulty. Byzantine workers
need not follow the master's instructions correctly, and might send malicious
incorrect (or faulty) information. The identity of the Byzantine workers
remains fixed throughout the learning process, and is unknown a priori to the
master. We propose two coding schemes, a deterministic scheme and a randomized
scheme, for guaranteeing exact fault-tolerance if $2f < n$. The coding schemes
use the concept of reactive redundancy for isolating Byzantine workers that
eventually send faulty information. We note that the computation efficiencies
of the schemes compare favorably with other (deterministic or randomized)
coding schemes, for exact fault-tolerance.

本文研究了在基于并行随机梯度下降（parallelized-SGD）算法的同步并行学习中的拜占庭容错问题。我们提出了两种编码方案，确定性编码方案和随机编码方案，用于保证在包含最多 f 个拜占庭错误的 n 个计算单元的体系结构下实现精确容错，且其计算效率比其他编码方案都好。