Robust distributed learning consists in achieving good learning performance
despite the presence of misbehaving workers. State-of-the-art (SOTA) robust
distributed gradient descent (Robust-DGD) methods, relying on robust
aggregation, have been proven to be optimal: Their learning error matches the
lower bound established under the standard heterogeneity model of $(G,
B)$-gradient dissimilarity. The learning guarantee of SOTA Robust-DGD cannot be
further improved when model initialization is done arbitrarily. However, we
show that it is possible to circumvent the lower bound, and improve the
learning performance, when the workers' gradients at model initialization are
assumed to be bounded. We prove this by proposing pre-aggregation clipping of
workers' gradients, using a novel scheme called adaptive robust clipping (ARC).
Incorporating ARC in Robust-DGD provably improves the learning, under the
aforementioned assumption on model initialization. The factor of improvement is
prominent when the tolerable fraction of misbehaving workers approaches the
breakdown point. ARC induces this improvement by constricting the search space,
while preserving the robustness property of the original aggregation scheme at
the same time. We validate this theoretical finding through exhaustive
experiments on benchmark image classification tasks.

通过引入新的方案，即自适应鲁棒剪裁 (ARC)，在模型初始化时限制工作人员梯度，从而改善鲁棒分布式梯度下降 (Robust-DGD) 的学习性能，特别是在可容忍的错误工作人员比例接近失效点时，该改进因子更为突出。