Recently, there has been a growing interest in text-to-speech (TTS) methods
that can be trained with minimal supervision by combining two types of discrete
speech representations and using two sequence-to-sequence tasks to decouple
TTS. To address the challenges associated with high dimensionality and waveform
distortion in discrete representations, we propose Diff-LM-Speech, which models
semantic embeddings into mel-spectrogram based on diffusion models and
introduces a prompt encoder structure based on variational autoencoders and
prosody bottlenecks to improve prompt representation capabilities.
Autoregressive language models often suffer from missing and repeated words,
while non-autoregressive frameworks face expression averaging problems due to
duration prediction models. To address these issues, we propose
Tetra-Diff-Speech, which designs a duration diffusion model to achieve diverse
prosodic expressions. While we expect the information content of semantic
coding to be between that of text and acoustic coding, existing models extract
semantic coding with a lot of redundant information and dimensionality
explosion. To verify that semantic coding is not necessary, we propose
Tri-Diff-Speech. Experimental results show that our proposed methods outperform
baseline methods. We provide a website with audio samples.

通过结合两种离散语音表示形式并使用两个序列到序列任务解耦合 TTS，最近在对可以以最低限度的监督进行训练的 TTS 方法中产生了越来越大的兴趣。我们提出了 Diff-LM-Speech，它在扩散模型的基础上将语义嵌入建模为基于 mel-spectrogram，并引入了基于变分自动编码器和韵律瓶颈的提示编码结构，以提高提示表示能力。我们还提出了 Tetra-Diff-Speech，通过设计一个持续时间扩散模型来实现多样化的韵律表达。同时，我们提出了 Tri-Diff-Speech 来验证语义编码的必要性。实验结果表明，我们的方法优于基准方法。我们提供了一个包含音频样本的网站。