We consider a distributed learning problem in which the computation is
carried out on a system consisting of a master node and multiple worker nodes.
In such systems, the existence of slow-running machines called stragglers will
cause a significant decrease in performance. Recently, coding theoretic
framework, which is named Gradient Coding (GC), for mitigating stragglers in
distributed learning has been established by Tandon et al. Most studies on GC
are aiming at recovering the gradient information completely assuming that the
Gradient Descent (GD) algorithm is used as a learning algorithm. On the other
hand, if the Stochastic Gradient Descent (SGD) algorithm is used, it is not
necessary to completely recover the gradient information, and its unbiased
estimator is sufficient for the learning. In this paper, we propose a
distributed SGD scheme using Low-Density Generator Matrix (LDGM) codes. In the
proposed system, it may take longer time than existing GC methods to recover
the gradient information completely, however, it enables the master node to
obtain a high-quality unbiased estimator of the gradient at low computational
cost and it leads to overall performance improvement.

该论文提出了一种使用低密度生成矩阵代码的分布式随机梯度下降方案，通过该方案可以在低计算成本下得到高质量的无偏梯度估计，进而改善整体性能。