Decentralized stochastic optimization methods have gained a lot of attention
recently, mainly because of their cheap per iteration cost, data locality, and
their communication-efficiency. In this paper we introduce a unified
convergence analysis that covers a large variety of decentralized SGD methods
which so far have required different intuitions, have different applications,
and which have been developed separately in various communities.
Our algorithmic framework covers local SGD updates and synchronous and
pairwise gossip updates on adaptive network topology. We derive universal
convergence rates for smooth (convex and non-convex) problems and the rates
interpolate between the heterogeneous (non-identically distributed data) and
iid-data settings, recovering linear convergence rates in many special cases,
for instance for over-parametrized models. Our proofs rely on weak assumptions
(typically improving over prior work in several aspects) and recover (and
improve) the best known complexity results for a host of important scenarios,
such as for instance coorperative SGD and federated averaging (local SGD).

这篇论文介绍了一种统一的收敛性分析方法，涵盖了许多分散式随机梯度下降方法，具有计算成本低、数据本地性和沟通效率等优点，并包括本地随机梯度下降更新和自适应网络拓扑上的同步和成对传递更新，我们推导了光滑（凸和非凸）问题的通用收敛率，并在不同的数据分布和 iid 数据设置下进行了插值。