There exist three approaches for multilingual and crosslingual automatic
speech recognition (MCL-ASR) - supervised pre-training with phonetic or
graphemic transcription, and self-supervised pre-training. We find that
pre-training with phonetic supervision has been underappreciated so far for
MCL-ASR, while conceptually it is more advantageous for information sharing
between different languages. This paper explores the approach of pre-training
with weakly phonetic supervision towards data-efficient MCL-ASR, which is
called Whistle. We relax the requirement of gold-standard human-validated
phonetic transcripts, and obtain International Phonetic Alphabet (IPA) based
transcription by leveraging the LanguageNet grapheme-to-phoneme (G2P) models.
We construct a common experimental setup based on the CommonVoice dataset,
called CV-Lang10, with 10 seen languages and 2 unseen languages. A set of
experiments are conducted on CV-Lang10 to compare, as fair as possible, the
three approaches under the common setup for MCL-ASR. Experiments demonstrate
the advantages of phoneme-based models (Whistle) for MCL-ASR, in terms of
speech recognition for seen languages, crosslingual performance for unseen
languages with different amounts of few-shot data, overcoming catastrophic
forgetting, and training efficiency.It is found that when training data is more
limited, phoneme supervision can achieve better results compared to subword
supervision and self-supervision, thereby providing higher data-efficiency. To
support reproducibility and promote future research along this direction, we
will release the code, models and data for the whole pipeline of Whistle at
this https URL upon publication.

本文研究了预训练与弱语音监督的方法，提出了一种名为 Whistle 的数据有效 MCL-ASR 方法，通过国际音标转写，构建了一个基于 CommonVoice 数据集的实验平台，实验证明基于音素的模型在多语言语音识别中具有更好的性能和高数据效率。