BriefGPT.xyz
May, 2023
平均加速随机梯度下降算法：有限样本速率和渐近正态性
Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality
HTML
PDF
Kejie Tang, Weidong Liu, Yichen Zhang
TL;DR
本研究分析了随机梯度下降与动量法在强凸设置下的有限样本收敛速度，并证明了 Polyak-averaging 版本的 SGDM 估算器的渐近正态性以及其与平均 SGD 的渐近等价性。
Abstract
stochastic gradient descent with momentum
(SGDM) has been widely used in many
machine learning
and statistical applications. Despite the observed empirical benefits of SGDM over traditional SGD, the theoretical u
→