Although automatic speech recognition (ASR) can perform well in common
non-overlapping environments, sustaining performance in multi-talker
overlapping speech recognition remains challenging. Recent research revealed
that ASR model's encoder captures different levels of information with
different layers -- the lower layers tend to have more acoustic information,
and the upper layers more linguistic. This inspires us to develop a Sidecar
separator to empower a well-trained ASR model for multi-talker scenarios by
separating the mixed speech embedding between two suitable layers. We
experimented with a wav2vec 2.0-based ASR model with a Sidecar mounted. By
freezing the parameters of the original model and training only the Sidecar
(8.7 M, 8.4% of all parameters), the proposed approach outperforms the previous
state-of-the-art by a large margin for the 2-speaker mixed LibriMix dataset,
reaching a word error rate (WER) of 10.36%; and obtains comparable results
(7.56%) for LibriSpeechMix dataset when limited training.

本研究提出了一种基于 Sidecar 分离器的多说话人语音识别方法，旨在提高 ASR 模型在多说话人情况下的识别效果，实验结果显示该方法优于现有最先进水平。