We propose a novel method to accelerate training and inference process of
recurrent neural network transducer (RNN-T) based on the guidance from a
co-trained connectionist temporal classification (CTC) model. We made a key
assumption that if an encoder embedding frame is classified as a blank frame by
the CTC model, it is likely that this frame will be aligned to blank for all
the partial alignments or hypotheses in RNN-T and it can be discarded from the
decoder input. We also show that this frame reduction operation can be applied
in the middle of the encoder, which result in significant speed up for the
training and inference in RNN-T. We further show that the CTC alignment, a
by-product of the CTC decoder, can also be used to perform lattice reduction
for RNN-T during training. Our method is evaluated on the Librispeech and
SpeechStew tasks. We demonstrate that the proposed method is able to accelerate
the RNN-T inference by 2.2 times with similar or slightly better word error
rates (WER).

通过协同训练的 CTC 模型的指导，我们提出了一种新颖的方法来加速基于递归神经网络传输器 (RNN-T) 的训练和推理过程，并通过在编码器中进行帧降维操作来减少计算量。在 Librispeech 和 SpeechStew 任务上评估表明，新方法可将 RNN-T 推理加速 2.2 倍而不影响单词错误率 (WER)。