BriefGPT.xyz
Jul, 2023
加权平均随机梯度下降：渐近正态性与最优性
Weighted Averaged Stochastic Gradient Descent: Asymptotic Normality and Optimality
HTML
PDF
Ziyang Wei, Wanrong Zhu, Wei Biao Wu
TL;DR
本文探讨了随机梯度下降算法的加速收敛方法，提出了一种自适应加权平均方案，并提供了非渐近收敛的统计保证和在线推断方法。最终的结论表明，该自适应加权平均方案不仅在统计率上是最优的，而且在非渐近收敛方面也具有有利的效果。
Abstract
stochastic gradient descent
(SGD) is one of the simplest and most popular algorithms in modern statistical and machine learning due to its computational and memory efficiency. Various
averaging schemes
have been
→