In this work, we propose a novel recurrent neural network (RNN) architecture.
The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of
stacking multiple recurrent layers by allowing and controlling signals flowing
from upper recurrent layers to lower layers using a global gating unit for each
pair of layers. The recurrent signals exchanged between layers are gated
adaptively based on the previous hidden states and the current input. We
evaluated the proposed GF-RNN with different types of recurrent units, such as
tanh, long short-term memory and gated recurrent units, on the tasks of
character-level language modeling and Python program evaluation. Our empirical
evaluation of different RNN units, revealed that in both tasks, the GF-RNN
outperforms the conventional approaches to build deep stacked RNNs. We suggest
that the improvement arises because the GF-RNN can adaptively assign different
layers to different timescales and layer-to-layer interactions (including the
top-down ones which are not usually present in a stacked RNN) by learning to
gate these interactions.

本研究提出一种新型的循环神经网络结构，即门控反馈循环神经网络，通过全局门控单元来控制和允许来自上层循环层到下层层的信号，改进了现有的多层循环层叠加方法，我们评估了不同类型的循环神经元，如 tanh，长短时记忆和门循环单元，在字符级语言建模和 Python 程序评估等任务上，实验证明 GF-RNN 优于传统方法构建深度堆叠循环神经网络，因为 GF-RNN 可以通过学习对这些交互进行门控从而自适应地将不同的层分配给不同的时间尺度和层间交互。