BriefGPT.xyz
Jul, 2019
深度网络中的权重空间对称性导致排列鞍点出现,在损失景观中通过等损谷相连
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape
HTML
PDF
Johanni Brea, Berfin Simsek, Bernd Illing, Wulfram Gerstner
TL;DR
该研究利用深度神经网络计算的几何方法,探讨网络层之间的置换对全局极小化及鞍点问题的影响及其数学意义。
Abstract
The
permutation symmetry
of neurons in each layer of a deep neural network gives rise not only to multiple equivalent
global minima
of the loss function, but also to first-order
→