We propose a flexible gradient tracking approach with adjustable computation
and communication steps for solving distributed stochastic optimization problem
over networks. The proposed method allows each node to perform multiple local
gradient updates and multiple inter-node communications in each round, aiming
to strike a balance between computation and communication costs according to
the properties of objective functions and network topology in non-i.i.d.
settings. Leveraging a properly designed Lyapunov function, we derive both the
computation and communication complexities for achieving arbitrary accuracy on
smooth and strongly convex objective functions. Our analysis demonstrates sharp
dependence of the convergence performance on graph topology and properties of
objective functions, highlighting the trade-off between computation and
communication. Numerical experiments are conducted to validate our theoretical
findings.

提出了一种灵活的梯度跟踪方法，用于解决非独立同分布情况下网络上的分布式随机优化问题，利用设计良好的李亚普诺夫函数，导出了计算和通信复杂度，以实现在光滑和强凸目标函数上的任意精度。

一种灵活的梯度追踪方法的计算 - 通讯权衡

On the Computation-Communication Trade-Off with A Flexible Gradient  Tracking Approach

We consider a distributed multi-agent network system where the goal is to
minimize a sum of convex objective functions of the agents subject to a common
convex constraint set. Each agent maintains an iterate sequence and
communicates the iterates to its neighbors. Then, each agent combines weighted
averages of the received iterates with its own iterate, and adjusts the iterate
by using subgradient information (known with stochastic errors) of its own
function and by projecting onto the constraint set.
The goal of this paper is to explore the effects of stochastic subgradient
errors on the convergence of the algorithm. We first consider the behavior of
the algorithm in mean, and then the convergence with probability 1 and in mean
square. We consider general stochastic errors that have uniformly bounded
second moments and obtain bounds on the limiting performance of the algorithm
in mean for diminishing and non-diminishing stepsizes. When the means of the
errors diminish, we prove that there is mean consensus between the agents and
mean convergence to the optimum function value for diminishing stepsizes. When
the mean errors diminish sufficiently fast, we strengthen the results to
consensus and convergence of the iterates to an optimal solution with
probability 1 and in mean square.

本文考虑了一个分布式的多智能体网络系统，旨在最小化代理的凸目标函数之和，并遵守共同的凸约束集。本文探讨了随机阶梯下降法的收敛性，并提出了一些保证收敛性的条件。