State-of-the-art large-scale universal speech models (USMs) show a decent
automatic speech recognition (ASR) performance across multiple domains and
languages. However, it remains a challenge for these models to recognize
overlapped speech, which is often seen in meeting conversations. We propose an
approach to adapt USMs for multi-talker ASR. We first develop an enhanced
version of serialized output training to jointly perform multi-talker ASR and
utterance timestamp prediction. That is, we predict the ASR hypotheses for all
speakers, count the speakers, and estimate the utterance timestamps at the same
time. We further introduce a lightweight adapter module to maintain the
multilingual property of the USMs even when we perform the adaptation with only
a single language. Experimental results obtained using the AMI and AliMeeting
corpora show that our proposed approach effectively transfers the USMs to a
strong multilingual multi-talker ASR model with timestamp prediction
capability.

该论文提出了一种使用改进的序列化输出训练和轻量级适配器模块来解决在会议对话中经常出现的多说话者自动语音识别（ASR）问题的方法，实验结果表明该方法有效地将 USMs 转换为具有时间戳预测能力的强大的多语言多说话人 ASR 模型