BriefGPT.xyz
May, 2020
通过加速梯度削减实现重尾噪声的随机优化
Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping
HTML
PDF
Eduard Gorbunov, Marina Danilova, Alexander Gasnikov
TL;DR
本文提出了一种新的加速随机一阶方法clipped-SSTM，该方法通过剪辑随机梯度结合特殊变体的随机梯度下降法，用于解决具有重尾分布噪声的光滑凸随机优化问题，并推导出了该方法的第一个高概率复杂度界限，证明了其优于同类方法。
Abstract
In this paper, we propose a new
accelerated stochastic first-order method
called clipped-SSTM for smooth convex
stochastic optimization
with heavy-tailed distributed noise in stochastic gradients and derive the f
→