BriefGPT.xyz
Mar, 2022
无捷径的深度学习:定制整流器塑造核心
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers
HTML
PDF
Guodong Zhang, Aleksandar Botev, James Martens
TL;DR
本研究提出了一种新的变换,完全兼容ReLUs的变种——Leaky ReLUs,并证明我们的方法可以实现与ResNets相当的准确性。
Abstract
Training very
deep neural networks
is still an extremely challenging task. The common solution is to use
shortcut connections
and
normalization l
→