The stochastic proximal gradient method is a powerful generalization of the
widely used stochastic gradient descent (SGD) method and has found numerous
applications in Machine Learning. However, it is notoriously known that this
method fails to converge in non-convex settings where the stochastic noise is
significant (i.e. when only small or bounded batch sizes are used). In this
paper, we focus on the stochastic proximal gradient method with Polyak
momentum. We prove this method attains an optimal convergence rate for
non-convex composite optimization problems, regardless of batch size.
Additionally, we rigorously analyze the variance reduction effect of the Polyak
momentum in the composite optimization setting and we show the method also
converges when the proximal step can only be solved inexactly. Finally, we
provide numerical experiments to validate our theoretical results.

该论文研究了在采用小型或有界批量大小时，在非凸设置中具有重要意义的随机近端梯度法，证明了该方法在非凸复合优化问题中达到最优的收敛速度，并且严格分析了 Polyak 动量在复合优化设置中的方差缩减效应，同时证明了在近似解决近端步骤的情况下，该方法仍然收敛，并通过数值实验验证了我们的理论结果。