We introduce and analyze stochastic optimization methods where the input to
each gradient update is perturbed by bounded noise. We show that this framework
forms the basis of a unified approach to analyze asynchronous implementations
of stochastic optimization algorithms.In this framework, asynchronous
stochastic optimization algorithms can be thought of as serial methods
operating on noisy inputs. Using our perturbed iterate framework, we provide
new analyses of the Hogwild! algorithm and asynchronous stochastic coordinate
descent, that are simpler than earlier analyses, remove many assumptions of
previous models, and in some cases yield improved upper bounds on the
convergence rates. We proceed to apply our framework to develop and analyze
KroMagnon: a novel, parallel, sparse stochastic variance-reduced gradient
(SVRG) algorithm. We demonstrate experimentally on a 16-core machine that the
sparse and parallel version of SVRG is in some cases more than four orders of
magnitude faster than the standard SVRG algorithm.

介绍和分析了一种基于随机优化算法和有界噪声扰动输入的方法，用于分析异步实现随机优化算法的统一方法，应用该方法开发和分析了一种新的稀疏随机方差降低梯度算法 KroMagnon，并在 16 核机器上进行了实验，表明稀疏和并行化版本的 SVRG 算法在某些情况下比标准 SVRG 算法快了四个数量级。