We present a CLSRIL-23, a self supervised learning based audio pre-trained
model which learns cross lingual speech representations from raw audio across
23 Indic languages. It is built on top of wav2vec 2.0 which is solved by
training a contrastive task over masked latent speech representations and
jointly learns the quantization of latents shared across all languages. We
compare the language wise loss during pretraining to compare effects of
monolingual and multilingual pretraining. Performance on some downstream
fine-tuning tasks for speech recognition is also compared and our experiments
show that multilingual pretraining outperforms monolingual training, in terms
of learning speech representations which encodes phonetic similarity of
languages and also in terms of performance on down stream tasks. A decrease of
5% is observed in WER and 9.5% in CER when a multilingual pretrained model is
used for finetuning in Hindi. All the code models are also open sourced.
CLSRIL-23 is a model trained on $23$ languages and almost 10,000 hours of audio
data to facilitate research in speech recognition for Indic languages. We hope
that new state of the art systems will be created using the self supervised
approach, especially for low resources Indic languages.

本研究提出了 CLSRIL-23，一种基于自监督学习的音频预训练模型，可以在 23 种印度语言的原始音频中学习跨语言的语音表示。在预训练期间，将语言特定的语音表示分别比较以比较单语和多语预训练的效果。我们的实验证明，多语预训练优于单语预训练，能够学习对语言进行编码并在下游任务中达到更好的性能。