Training state-of-the-art Automated Speech Recognition (ASR) models typically
requires a substantial amount of transcribed speech. In this work, we
demonstrate that a modality-matched joint speech and text model can be
leveraged to train a massively multilingual ASR model without any supervised
(manually transcribed) speech for some languages. This paper explores the use
of jointly learnt speech and text representations in a massively multilingual,
zero supervised speech, real-world setting to expand the set of languages
covered by ASR with only unlabeled speech and text in the target languages.
Using the FLEURS dataset, we define the task to cover $102$ languages, where
transcribed speech is available in $52$ of these languages and can be used to
improve end-to-end ASR quality on the remaining $50$. First, we show that by
combining speech representations with byte-level text representations and use
of language embeddings, we can dramatically reduce the Character Error Rate
(CER) on languages with no supervised speech from 64.8\% to 30.8\%, a relative
reduction of 53\%. Second, using a subset of South Asian languages we show that
Maestro-U can promote knowledge transfer from languages with supervised speech
even when there is limited to no graphemic overlap. Overall, Maestro-U closes
the gap to oracle performance by 68.5\% relative and reduces the CER of 19
languages below 15\%.

在这篇论文中，我们演示了利用匹配模态的联合语音和文本模型可以训练出一个巨大的多语言自动语音识别模型，即使对于某些语言没有使用受控制的语音作为训练数据。我们还展示了 Maestro-U 的表现，它可以使错误率降低 68.5％，并将 19 种语言的 CER 降到 15％以下。