Stream fusion, also known as system combination, is a common technique in
automatic speech recognition for traditional hybrid hidden Markov model
approaches, yet mostly unexplored for modern deep neural network end-to-end
model architectures. Here, we investigate various fusion techniques for the
all-attention-based encoder-decoder architecture known as the transformer,
striving to achieve optimal fusion by investigating different fusion levels in
an example single-microphone setting with fusion of standard magnitude and
phase features. We introduce a novel multi-encoder learning method that
performs a weighted combination of two encoder-decoder multi-head attention
outputs only during training. Employing then only the magnitude feature encoder
in inference, we are able to show consistent improvement on Wall Street Journal
(WSJ) with language model and on Librispeech, without increase in runtime or
parameters. Combining two such multi-encoder trained models by a simple late
fusion in inference, we achieve state-of-the-art performance for
transformer-based models on WSJ with a significant WER reduction of 19%
relative compared to the current benchmark approach.

通过研究多种融合技术和多编码器学习方法，以实现对变压器架构的最佳融合，仅在训练期间对两个编码器 - 解码器组合的多头注意权重输出进行加权组合，在推理时使用幅度特征编码器，成功地在 WSJ 上展示了一致的改进，并通过简单的后期融合在 WSJ 上取得了状态 - of-the-art 的表现。