BriefGPT.xyz
May, 2021
使用层归一化重新思考Transformer和ResNet中的跳跃连接
Rethinking Skip Connection with Layer Normalization in Transformers and ResNets
HTML
PDF
Fenglin Liu, Xuancheng Ren, Zhiyuan Zhang, Xu Sun, Yuexian Zou
TL;DR
研究了跳跃连接技术中规模因子对其效率的影响,提出了递归应用带有层归一化的跳跃连接技术可以显著提高性能并在各种任务包括机器翻译和图像分类技术中具有很好的普适性。
Abstract
skip connection
, is a widely-used technique to improve the
performance
and the convergence of deep neural networks, which is believed to relieve the difficulty in
→