We study the fundamental limits to communication-efficient distributed
methods for convex learning and optimization, under different assumptions on
the information available to individual machines, and the types of functions
considered. We identify cases where existing algorithms are already worst-case
optimal, as well as cases where room for further improvement is still possible.
Among other things, our results indicate that without similarity between the
local objective functions (due to statistical data similarity or otherwise)
many communication rounds may be required, even if the machines have unbounded
computational power.

研究了分布式方法在凸学习与优化中所需要的通信效率的根本限制，在不同信息假设和函数类型条件下找到了现有算法已达到最坏复杂度的情况，同时也指出了仍有改进的余地，说明了当本地目标函数没有相似之处时（由于统计数据相似或其他原因），即使设备具有无限计算能力，也可能需要多次通信往返。

分布式凸优化的通信复杂度

Communication Complexity of Distributed Convex Learning and Optimization

In decentralized consensus optimization, a connected network of agents
collaboratively minimize the sum of their local objective functions over a
common decision variable, where their information exchange is restricted
between the neighbors. To this end, one can first obtain a problem
reformulation and then apply the alternating direction method of multipliers
(ADMM). The method applies iterative computation at the individual agents and
information exchange between the neighbors. This approach has been observed to
converge quickly and deemed powerful. This paper establishes its linear
convergence rate for decentralized consensus optimization problem with strongly
convex local objective functions. The theoretical convergence rate is
explicitly given in terms of the network topology, the properties of local
objective functions, and the algorithm parameter. This result is not only a
performance guarantee but also a guideline toward accelerating the ADMM
convergence.

本文研究了具有强凸局部目标函数的去中心化一致性优化问题，并建立了其线性收敛速率及加速的指导原则，其中采用了交替方向乘子方法 (ADMM) 来解决该问题，该方法可在单个代理处进行迭代计算并在邻居之间进行信息交换。