BriefGPT.xyz
Aug, 2017
重新审视激活正则化对于语言循环神经网络的影响
Revisiting Activation Regularization for Language RNNs
HTML
PDF
Stephen Merity, Bryan McCann, Richard Socher
TL;DR
本文通过使用传统正则化技术L2规则化和缓慢规则化来提高RNN对于语言建模任务的性能,而这两种技术只需要对现有RNN架构进行最小修改,即可获得与复杂正则化技术或自定义单元结构相媲美或更好的性能表现,并且这些技术可以在现有的优化LSTM实现上无需进行任何修改。
Abstract
recurrent neural networks
(RNNs) serve as a fundamental building block for many sequence tasks across natural language processing. Recent research has focused on recurrent dropout techniques or custom
rnn
cells i
→