BriefGPT.xyz
Aug, 2020
浅层ReLU模型中Hessian的分析特性：一段关于对称性的故事
Analytic Characterization of the Hessian in Shallow ReLU Models: A Tale of Symmetry
HTML
PDF
Yossi Arjevani, Michael Field
TL;DR
论文研究用两层 ReLU 神经网络中 Hessian 矩阵的对称性状结构及其在寻找拟最小值时的作用，指出 Hessian 矩阵的本征值存在极度不平衡的现象，为统计推广提供了重要参考。
Abstract
We consider the optimization problem associated with fitting
two-layers relu networks
with $k$ neurons. We leverage the rich
symmetry structure
to analytically characterize the
→