In this paper, we present a Diffusion GAN based approach (Prosodic Diff-TTS)
to generate the corresponding high-fidelity speech based on the style
description and content text as an input to generate speech samples within only
4 denoising steps. It leverages the novel conditional prosodic layer
normalization to incorporate the style embeddings into the multi head attention
based phoneme encoder and mel spectrogram decoder based generator architecture
to generate the speech. The style embedding is generated by fine tuning the
pretrained BERT model on auxiliary tasks such as pitch, speaking speed,
emotion,gender classifications. We demonstrate the efficacy of our proposed
architecture on multi-speaker LibriTTS and PromptSpeech datasets, using
multiple quantitative metrics that measure generated accuracy and MOS.

使用基于扩散生成对抗网络的方法（Prosodic Diff-TTS），通过将风格描述和内容文本作为输入生成仅经过 4 个去噪步骤的高保真语音样本。它利用了新颖的条件韵律层归一化，将风格嵌入结合到基于多头注意力的音素编码器和基于梅尔频谱图解码器的生成器体系结构中来生成语音。风格嵌入是通过在辅助任务上微调预训练的 BERT 模型，如音高、说话速度、情感和性别分类等生成的。我们使用多种量化指标对我们提出的架构在多说话人 LibriTTS 和 PromptSpeech 数据集上的有效性进行了验证，以衡量生成的准确性和 MOS 值。