This paper considers the decentralized composite optimization problem. We
propose a novel decentralized variance-reduction proximal-gradient algorithmic
framework, called PMGT-VR, which is based on a combination of several
techniques including multi-consensus, gradient tracking, and variance
reduction. The proposed framework relies on an imitation of centralized
algorithms and we demonstrate that algorithms under this framework achieve
convergence rates similar to that of their centralized counterparts. We also
describe and analyze two representative algorithms, PMGT-SAGA and PMGT-LSVRG,
and compare them to existing state-of-the-art proximal algorithms. To the best
of our knowledge, PMGT-VR is the first linearly convergent decentralized
stochastic algorithm that can solve decentralized composite optimization
problems. Numerical experiments are provided to demonstrate the effectiveness
of the proposed algorithms.

本研究提出了一种新的分散化随机算法框架 PMGT-VR，它结合了多种技术，包括多一致性、梯度跟踪和方差缩减。该框架依赖于中心化算法的模仿，并且证明算法在该框架下的收敛速度类似于其中心化对应物。我们还介绍和分析了两个代表性算法 PMGT-SAGA 和 PMGT-LSVRG，并将其与现有的基于 Proximal 的最先进算法进行了比较。在我们所知道的范围内，PMGT-VR 是第一个可以解决分散复合优化问题的线性收敛分散随机算法。实证实验证明了所提出算法的有效性。