BriefGPT.xyz
Dec, 2018
Batch Normalization 自动调速的理论分析
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization
HTML
PDF
Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu
TL;DR
本篇论文为Batch Normalization提供理论支持：即使在不同的学习速率下，通过gradient descent求解, BN仍然可以使得收敛的速度达到最佳水平。
Abstract
batch normalization
(BN) has become a cornerstone of
deep learning
across diverse architectures, appearing to help
optimization
as well as
→