Deep neural networks have recently been shown to achieve highly competitive
performance in many computer vision tasks due to their abilities of exploring
in a much larger hypothesis space. However, since most deep architectures like
stacked RNNs tend to suffer from the vanishing-gradient and overfitting
problems, their effects are still understudied in many NLP tasks. Inspired by
this, we propose a novel multi-layer RNN model called densely connected
bidirectional long short-term memory (DC-Bi-LSTM) in this paper, which
essentially represents each layer by the concatenation of its hidden state and
all preceding layers' hidden states, followed by recursively passing each
layer's representation to all subsequent layers. We evaluate our proposed model
on five benchmark datasets of sentence classification. DC-Bi-LSTM with depth up
to 20 can be successfully trained and obtain significant improvements over the
traditional Bi-LSTM with the same or even less parameters. Moreover, our model
has promising performance compared with the state-of-the-art approaches.

本文提出了一种新的多层 RNN 模型 —— 密集连接的双向长短期记忆网络 (DC-Bi-LSTM)，并在五个句子分类基准数据集上进行了评估。结果显示，相比同等或更少的参数的传统双向 LSTM，深度为 20 的 DC-Bi-LSTM 模型可以成功训练，并获得了显著的改进。此外，与最先进方法相比，我们的模型表现具有很好的潜力。