BriefGPT.xyz
Oct, 2023
小批次随机梯度下降和局部随机梯度下降的稳定性与泛化能力
Stability and Generalization for Minibatch SGD and Local SGD
HTML
PDF
Yunwen Lei, Tao Sun, Mingrui Liu
TL;DR
通过稳定性和泛化性分析研究了小批量随机梯度下降和本地随机梯度下降的学习能力，从而证明它们可以实现线性加速度以达到最佳风险界限。
Abstract
The increasing scale of
data
propels the popularity of leveraging
parallelism
to speed up the optimization.
minibatch stochastic gradient descent
→