The article examines in some detail the convergence rate and
mean-square-error performance of momentum stochastic gradient methods in the
constant step-size and slow adaptation regime. The results establish that
momentum methods are equivalent to the standard stochastic gradient method with
a re-scaled (larger) step-size value. The size of the re-scaling is determined
by the value of the momentum parameter. The equivalence result is established
for all time instants and not only in steady-state. The analysis is carried out
for general strongly convex and smooth risk functions, and is not limited to
quadratic risks. One notable conclusion is that the well-known bene ts of
momentum constructions for deterministic optimization problems do not
necessarily carry over to the adaptive online setting when small constant
step-sizes are used to enable continuous adaptation and learn- ing in the
presence of persistent gradient noise. From simulations, the equivalence
between momentum and standard stochastic gradient methods is also observed for
non-differentiable and non-convex problems.

研究了动量随机梯度法在常数步长和慢适应区域的收敛速度和均方误差性能，结果表明动量方法相当于具有重新缩放（更大）步长值的标准随机梯度法，重新缩放的大小由动量参数的值确定。分析适用于一般的强凸和光滑风险函数，而不限于二次风险，发现动量构建在确定性优化问题中的优点并不一定适用于小常数步长的自适应在线设置中，以使根据持续的梯度噪声进行连续适应和学习。模拟结果表明，动量和标准随机梯度方法在非可微和非凸问题中也是等价的。