BriefGPT.xyz
Dec, 2018
深度学习中方差缩减优化算法的无效性
On the Ineffectiveness of Variance Reduced Optimization for Deep Learning
HTML
PDF
Aaron Defazio, Léon Bottou
TL;DR
本文探讨了随机方差缩小技术在优化中的应用，研究发现在训练现代深度神经网络中，由于遇到难解的非凸优化问题，直接应用SVRG技术等方法效果不佳。
Abstract
The application of
stochastic variance reduction
to
optimization
has shown remarkable recent theoretical and practical success. The applicability of these techniques to the hard
→