We resolve the min-max complexity of distributed stochastic convex
optimization (up to a log factor) in the intermittent communication setting,
where $M$ machines work in parallel over the course of $R$ rounds of
communication to optimize the objective, and during each round of
communication, each machine may sequentially compute $K$ stochastic gradient
estimates. We present a novel lower bound with a matching upper bound that
establishes an optimal algorithm.

研究分布式随机凸优化的最小最大复杂度，在间歇通信设置下提出了一种新的下限和上限算法，以确定最佳算法。

分布式随机凸优化的极小 - 极大复杂度及间歇通信

The Min-Max Complexity of Distributed Stochastic Convex Optimization  with Intermittent Communication

We study distributed stochastic convex optimization under the delayed
gradient model where the server nodes perform parameter updates, while the
worker nodes compute stochastic gradients. We discuss, analyze, and experiment
with a setup motivated by the behavior of real-world distributed computation
networks, where the machines are differently slow at different time. Therefore,
we allow the parameter updates to be sensitive to the actual delays
experienced, rather than to worst-case bounds on the maximum delay. This
sensitivity leads to larger stepsizes, that can help gain rapid initial
convergence without having to wait too long for slower machines, while
maintaining the same asymptotic complexity. We obtain encouraging improvements
to overall convergence for distributed experiments on real datasets with up to
billions of examples and features.

该研究探讨并分析了实际分布式计算网络情况下的延迟梯度模型，允许参数更新对实际延迟进行敏感处理以产生更大的步长，从而在保持相同渐近复杂度的同时实现更快收敛，针对包含数十亿个数据和特征的真实数据集展开分布式实验，并取得鼓舞人心的改进。