Recently, leveraging BERT pre-training to improve the phoneme encoder in text
to speech (TTS) has drawn increasing attention. However, the works apply
pre-training with character-based units to enhance the TTS phoneme encoder,
which is inconsistent with the TTS fine-tuning that takes phonemes as input.
Pre-training only with phonemes as input can alleviate the input mismatch but
lack the ability to model rich representations and semantic information due to
limited phoneme vocabulary. In this paper, we propose MixedPhoneme BERT, a
novel variant of the BERT model that uses mixed phoneme and sup-phoneme
representations to enhance the learning capability. Specifically, we merge the
adjacent phonemes into sup-phonemes and combine the phoneme sequence and the
merged sup-phoneme sequence as the model input, which can enhance the model
capacity to learn rich contextual representations. Experiment results
demonstrate that our proposed Mixed-Phoneme BERT significantly improves the TTS
performance with 0.30 CMOS gain compared with the FastSpeech 2 baseline. The
Mixed-Phoneme BERT achieves 3x inference speedup and similar voice quality to
the previous TTS pre-trained model PnG BERT

本文提出了使用混合音素和超音素表示来增强 BERT 模型的学习能力的 MixedPhoneme BERT，该模型将相邻的音素合并为超音素，并将音素序列和超音素序列组合作为模型输入，从而在文本到语音合成中获得了显着的性能提升并且速度快 3 倍。