BriefGPT.xyz
Feb, 2024
增强随机梯度下降：更快收敛的统一框架和新的加速方法
Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
HTML
PDF
Yichuan Deng, Zhao Song, Chiwun Yang
TL;DR
基于SGD，本文提出了一种统一框架来解决随机优化中非凸条件下的收敛分析问题，并发现了两种插入加速方法：拒绝加速和随机向量加速，理论上证明这两种方法可以直接提高收敛速度。
Abstract
Based on
sgd
, previous works have proposed many algorithms that have improved convergence speed and generalization in
stochastic optimization
, such as SGDm, AdaGrad, Adam, etc. However, their
→