BriefGPT.xyz
Jun, 2020
使用标准化层训练的ReLU神经网络的优化理论
Optimization Theory for ReLU Neural Networks Trained with Normalization Layers
HTML
PDF
Yonatan Dukler, Quanquan Gu, Guido Montúfar
TL;DR
本文研究了引入规范化层对神经网络的全局收敛速度和优化效果的影响，证明了在引入Weight Normalization后，具有ReLU激活函数的两层神经网络可以加速收敛。
Abstract
The success of
deep neural networks
is in part due to the use of
normalization layers
.
normalization layers
like Batch Normalization, Laye
→