We introduce a multilingual speaker change detection model (USM-SCD) that can
simultaneously detect speaker turns and perform ASR for 96 languages. This
model is adapted from a speech foundation model trained on a large quantity of
supervised and unsupervised data, demonstrating the utility of fine-tuning from
a large generic foundation model for a downstream task. We analyze the
performance of this multilingual speaker change detection model through a
series of ablation studies. We show that the USM-SCD model can achieve more
than 75% average speaker change detection F1 score across a test set that
consists of data from 96 languages. On American English, the USM-SCD model can
achieve an 85.8% speaker change detection F1 score across various public and
internal test sets, beating the previous monolingual baseline model by 21%
relative. We also show that we only need to fine-tune one-quarter of the
trainable model parameters to achieve the best model performance. The USM-SCD
model exhibits state-of-the-art ASR quality compared with a strong public ASR
baseline, making it suitable to handle both tasks with negligible additional
computational cost.

我们引入了一种多语种说话人变更检测模型（USM-SCD），可以同时检测 96 种语言的说话人转换并进行 ASR。通过一系列消融研究，我们分析了这种多语种说话人变更检测模型的性能，并证明了从大规模通用基础模型进行微调对下游任务的实用性。USM-SCD 模型在包含 96 种语言数据的测试集上能够实现超过 75% 的平均说话人变更检测 F1 得分。在美式英语上，与各种公开和内部测试集相比，USM-SCD 模型能够实现 85.8% 的说话人变更检测 F1 得分，相对于之前的单语言基准模型提高了 21%。我们还表明只需要微调一个四分之一的可训练模型参数就能实现最佳模型性能。USM-SCD 模型在与强大的公开 ASR 基线相比具有最先进的 ASR 质量，适用于同时处理这两个任务并且计算成本几乎可以忽略。