Text-to-speech (TTS) methods have shown promising results in voice cloning,
but they require a large number of labeled text-speech pairs.
Minimally-supervised speech synthesis decouples TTS by combining two types of
discrete speech representations(semantic \& acoustic) and using two
sequence-to-sequence tasks to enable training with minimal supervision.
However, existing methods suffer from information redundancy and dimension
explosion in semantic representation, and high-frequency waveform distortion in
discrete acoustic representation. Autoregressive frameworks exhibit typical
instability and uncontrollability issues. And non-autoregressive frameworks
suffer from prosodic averaging caused by duration prediction models. To address
these issues, we propose a minimally-supervised high-fidelity speech synthesis
method, where all modules are constructed based on the diffusion models. The
non-autoregressive framework enhances controllability, and the duration
diffusion model enables diversified prosodic expression. Contrastive
Token-Acoustic Pretraining (CTAP) is used as an intermediate semantic
representation to solve the problems of information redundancy and dimension
explosion in existing semantic coding methods. Mel-spectrogram is used as the
acoustic representation. Both semantic and acoustic representations are
predicted by continuous variable regression tasks to solve the problem of
high-frequency fine-grained waveform distortion. Experimental results show that
our proposed method outperforms the baseline method. We provide audio samples
on our website.

我们提出了一种基于扩散模型的最小监督高保真语音合成方法，其中所有模块均基于扩散模型构建，非自回归框架增强了可控性，持续时间扩散模型实现了多样化的韵律表达。