We present a comprehensive study of deep bidirectional long short-term memory
(LSTM) recurrent neural network (RNN) based acoustic models for automatic
speech recognition (ASR). We study the effect of size and depth and train
models of up to 8 layers. We investigate the training aspect and study
different variants of optimization methods, batching, truncated
backpropagation, different regularization techniques such as dropout and $L_2$
regularization, and different gradient clipping variants.
The major part of the experimental analysis was performed on the Quaero
corpus. Additional experiments also were performed on the Switchboard corpus.
Our best LSTM model has a relative improvement in word error rate of over 14\%
compared to our best feed-forward neural network (FFNN) baseline on the Quaero
task. On this task, we get our best result with an 8 layer bidirectional LSTM
and we show that a pretraining scheme with layer-wise construction helps for
deep LSTMs.
Finally we compare the training calculation time of many of the presented
experiments in relation with recognition performance.
All the experiments were done with RETURNN, the RWTH extensible training
framework for universal recurrent neural networks in combination with RASR, the
RWTH ASR toolkit.

本研究对基于双向长短时记忆（LSTM）循环神经网络（RNN）的自动语音识别（ASR）进行了全面的研究，探索了深度、大小、优化方法等不同因素的影响，并通过实验分析在 Quaero 语料库上实现了提高词错误率的效果，并比较了不同的培训计算时间。