In this paper, we consider a decentralized learning problem in the presence
of stragglers. Although gradient coding techniques have been developed for
distributed learning to evade stragglers, where the devices send encoded
gradients with redundant training data, it is difficult to apply those
techniques directly to decentralized learning scenarios. To deal with this
problem, we propose a new gossip-based decentralized learning method with
gradient coding (GOCO). In the proposed method, to avoid the negative impact of
stragglers, the parameter vectors are updated locally using encoded gradients
based on the framework of stochastic gradient coding and then averaged in a
gossip-based manner. We analyze the convergence performance of GOCO for
strongly convex loss functions. And we also provide simulation results to
demonstrate the superiority of the proposed method in terms of learning
performance compared with the baseline methods.

在本论文中，我们考虑了分布式学习中存在滞后问题的分散化学习问题。虽然梯度编码技术已经应用于分布式学习以规避滞后问题，其中设备发送带有冗余训练数据的编码梯度，但是直接应用这些技术到分散化学习场景中比较困难。为了解决这个问题，我们提出了一种基于流言蜚语的带有梯度编码的分散化学习方法（GOCO）。在该方法中，为了避免滞后问题的负面影响，参数向量使用基于随机梯度编码框架的编码梯度进行本地更新，并以基于流言蜚语的方式进行平均。我们分析了 GOCO 在强凸损失函数下的收敛性能，并通过仿真结果证明了所提出方法在学习性能方面相对于基准方法的优越性。