Neural Text-to-Speech (TTS) systems find broad applications in voice
assistants, e-learning, and audiobook creation. The pursuit of modern models,
like Diffusion Models (DMs), holds promise for achieving high-fidelity,
real-time speech synthesis. Yet, the efficiency of multi-step sampling in
Diffusion Models presents challenges. Efforts have been made to integrate GANs
with DMs, speeding up inference by approximating denoising distributions, but
this introduces issues with model convergence due to adversarial training. To
overcome this, we introduce CM-TTS, a novel architecture grounded in
consistency models (CMs). Drawing inspiration from continuous-time diffusion
models, CM-TTS achieves top-quality speech synthesis in fewer steps without
adversarial training or pre-trained model dependencies. We further design
weighted samplers to incorporate different sampling positions into model
training with dynamic probabilities, ensuring unbiased learning throughout the
entire training process. We present a real-time mel-spectrogram generation
consistency model, validated through comprehensive evaluations. Experimental
results underscore CM-TTS's superiority over existing single-step speech
synthesis systems, representing a significant advancement in the field.

通过引入连续时间扩散模型，CM-TTS 在不需要对抗训练或预训练模型依赖的情况下，在较少的步骤中实现了高质量的语音合成，通过动态概率设计加权采样器来确保整个训练过程的无偏学习。实验证实了 CM-TTS 相对于现有的单步语音合成系统的卓越性能，代表了该领域的重大进展。