We present XPhoneBERT, the first multilingual model pre-trained to learn
phoneme representations for the downstream text-to-speech (TTS) task. Our
XPhoneBERT has the same model architecture as BERT-base, trained using the
RoBERTa pre-training approach on 330M phoneme-level sentences from nearly 100
languages and locales. Experimental results show that employing XPhoneBERT as
an input phoneme encoder significantly boosts the performance of a strong
neural TTS model in terms of naturalness and prosody and also helps produce
fairly high-quality speech with limited training data. We publicly release our
pre-trained XPhoneBERT with the hope that it would facilitate future research
and downstream TTS applications for multiple languages. Our XPhoneBERT model is
available at this https URL

本研究介绍了 XPhoneBERT，这是第一个多语种模型，经过预训练学习用于下游文本转语音任务的音素表示。我们使用 RoBERTa 预训练方法对近 100 种语言和地区的 330M 个音素级句子进行了训练。实验结果表明，将 XPhoneBERT 用作输入音素编码器，可以显著提高强神经 TTS 模型的自然度和韵律性，并且有助于在有限的训练数据下产生高质量的语音。我们公开发布了预训练的 XPhoneBERT，希望能促进未来多语言下游 TTS 应用研究。