BriefGPT.xyz
Jun, 2014
高维非凸优化中鞍点问题的识别与攻克
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
HTML
PDF
Yann Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli...
TL;DR
本文根据统计物理学、随机矩阵理论、神经网络理论和实证证据,证明高维问题中鞍点而非局部极小值点是造成误差函数最小值难以求解的主要原因,因此,提出了一种新的二阶优化方法——无鞍牛顿法,用以快速逃脱高维鞍点并优化深度或递归神经网络。
Abstract
A central challenge to many fields of science and engineering involves minimizing
non-convex error functions
over continuous, high dimensional spaces.
gradient descent
or quasi-Newton methods are almost ubiquitou
→