In this paper, we present StyleTTS 2, a text-to-speech (TTS) model that
leverages style diffusion and adversarial training with large speech language
models (SLMs) to achieve human-level TTS synthesis. StyleTTS 2 differs from its
predecessor by modeling styles as a latent random variable through diffusion
models to generate the most suitable style for the text without requiring
reference speech, achieving efficient latent diffusion while benefiting from
the diverse speech synthesis offered by diffusion models. Furthermore, we
employ large pre-trained SLMs, such as WavLM, as discriminators with our novel
differentiable duration modeling for end-to-end training, resulting in improved
speech naturalness. StyleTTS 2 surpasses human recordings on the single-speaker
LJSpeech dataset and matches it on the multispeaker VCTK dataset as judged by
native English speakers. Moreover, when trained on the LibriTTS dataset, our
model outperforms previous publicly available models for zero-shot speaker
adaptation. This work achieves the first human-level TTS on both single and
multispeaker datasets, showcasing the potential of style diffusion and
adversarial training with large SLMs. The audio demos and source code are
available at this https URL

本文提出了 StyleTTS2，它是一个使用了样式扩散和对抗训练技术以及大型语音语言模型的文本转语音模型，它能够有效地进行潜在扩散，实现单个和多个说话人的人类级 TTS 合成。