Edge computing has recently emerged as a promising paradigm to boost the
performance of distributed learning by leveraging the distributed resources at
edge nodes. Architecturally, the introduction of edge nodes adds an additional
intermediate layer between the master and workers in the original distributed
learning systems, potentially leading to more severe straggler effect.
Recently, coding theory-based approaches have been proposed for stragglers
mitigation in distributed learning, but the majority focus on the conventional
workers-master architecture. In this paper, along a different line, we
investigate the problem of mitigating the straggler effect in hierarchical
distributed learning systems with an additional layer composed of edge nodes.
Technically, we first derive the fundamental trade-off between the
computational loads of workers and the stragglers tolerance. Then, we propose a
hierarchical gradient coding framework, which provides better stragglers
mitigation, to achieve the derived computational trade-off. To further improve
the performance of our framework in heterogeneous scenarios, we formulate an
optimization problem with the objective of minimizing the expected execution
time for each iteration in the learning process. We develop an efficient
algorithm to mathematically solve the problem by outputting the optimum
strategy. Extensive simulation results demonstrate the superiority of our
schemes compared with conventional solutions.

通过引入边缘节点作为额外的中间层，本文探讨了在分层分布式学习系统中缓解滞后效应的问题，首先推导了工作节点的计算负载和滞后者容忍度之间的基本权衡关系，然后提出了一种层级渐进编码框架，以实现所得到的计算权衡。为了在异构场景中进一步提高我们的框架性能，我们制定了一个优化问题，旨在最小化学习过程中每次迭代的预期执行时间，并设计了一种高效算法来数学求解该问题，得到最优策略。通过大量模拟结果，证明了我们的方案相对于传统解决方案的优越性。