Recently, fully recurrent neural network (RNN) based end-to-end models have
been proven to be effective for multi-speaker speech recognition in both the
single-channel and multi-channel scenarios. In this work, we explore the use of
Transformer models for these tasks by focusing on two aspects. First, we
replace the RNN-based encoder-decoder in the speech recognition model with a
Transformer architecture. Second, in order to use the Transformer in the
masking network of the neural beamformer in the multi-channel case, we modify
the self-attention component to be restricted to a segment rather than the
whole sequence in order to reduce computation. Besides the model architecture
improvements, we also incorporate an external dereverberation preprocessing,
the weighted prediction error (WPE), enabling our model to handle reverberated
signals. Experiments on the spatialized wsj1-2mix corpus show that the
Transformer-based models achieve 40.9% and 25.6% relative WER reduction, down
to 12.1% and 6.4% WER, under the anechoic condition in single-channel and
multi-channel tasks, respectively, while in the reverberant case, our methods
achieve 41.5% and 13.8% relative WER reduction, down to 16.5% and 15.2% WER.

本文研究使用 Transformer 模型替代基于循环神经网络的编码器 - 解码器模型，应用于多说话者语音识别和神经束形成器中的遮盖网络，得以有效处理混响信号，并加入外部去混响预处理方法进行对比试验。实验证明，在单通道和多通道任务下，基于 Transformer 的模型相对错误率降低达 40.9% 和 25.6%，在混响环境中的相对错误率降低达 41.5% 和 13.8%。