State-of-the-art non-autoregressive text-to-speech (TTS) models based on
FastSpeech 2 can efficiently synthesise high-fidelity and natural speech. For
expressive speech datasets however, we observe characteristic audio
distortions. We demonstrate that such artefacts are introduced to the vocoder
reconstruction by over-smooth mel-spectrogram predictions, which are induced by
the choice of mean-squared-error (MSE) loss for training the mel-spectrogram
decoder. With MSE loss FastSpeech 2 is limited to learn conditional averages of
the training distribution, which might not lie close to a natural sample if the
distribution still appears multimodal after all conditioning signals. To
alleviate this problem, we introduce TVC-GMM, a mixture model of
Trivariate-Chain Gaussian distributions, to model the residual multimodality.
TVC-GMM reduces spectrogram smoothness and improves perceptual audio quality in
particular for expressive datasets as shown by both objective and subjective
evaluation.

本篇论文通过引入 TVC-GMM，一种 Trivariate-Chain 高斯分布的混合模型，将过度平滑的 mel-spectrogram 有条件解码器进行建模，从而改善表达数据集中的音频质量。

通过建模残差多模态实现鲁棒的 FastSpeech 2

Towards Robust FastSpeech 2 by Modelling Residual Multimodality

Recently, we have seen an increasing interest in the area of speech-to-text
translation. This has led to astonishing improvements in this area. In
contrast, the activities in the area of speech-to-speech translation is still
limited, although it is essential to overcome the language barrier. We believe
that one of the limiting factors is the availability of appropriate training
data. We address this issue by creating LibriS2S, to our knowledge the first
publicly available speech-to-speech training corpus between German and English.
For this corpus, we used independently created audio for German and English
leading to an unbiased pronunciation of the text in both languages. This allows
the creation of a new text-to-speech and speech-to-speech translation model
that directly learns to generate the speech signal based on the pronunciation
of the source language. Using this created corpus, we propose Text-to-Speech
models based on the example of the recently proposed FastSpeech 2 model that
integrates source language information. We do this by adapting the model to
take information such as the pitch, energy or transcript from the source speech
as additional input.

该研究找出了语音到语音翻译领域所面临的主要限制是缺乏合适的培训数据。为了解决这个问题，研究人员创建了一个公开可用的语音到语音训练语料库 ——LibriS2S。基于这个语料库，提出了基于 FastSpeech 2 模型的新文本到语音模型，以及如何使模型直接根据源语言的发音来生成语音信号。

LibriS2S: 一份德英语音到语音翻译语料库

LibriS2S: A German-English Speech-to-Speech Translation Corpus

In this paper, we propose a text-to-speech (TTS)-driven data augmentation
method for improving the quality of a non-autoregressive (AR) TTS system.
Recently proposed non-AR models, such as FastSpeech 2, have successfully
achieved fast speech synthesis system. However, their quality is not
satisfactory, especially when the amount of training data is insufficient. To
address this problem, we propose an effective data augmentation method using a
well-designed AR TTS system. In this method, large-scale synthetic corpora
including text-waveform pairs with phoneme duration are generated by the AR TTS
system and then used to train the target non-AR model. Perceptual listening
test results showed that the proposed method significantly improved the quality
of the non-AR TTS system. In particular, we augmented five hours of a training
database to 179 hours of a synthetic one. Using these databases, our TTS system
consisting of a FastSpeech 2 acoustic model with a Parallel WaveGAN vocoder
achieved a mean opinion score of 3.74, which is 40% higher than that achieved
by the conventional method.

本文提出了一种基于 TTS 的数据增强方法，用于改进非自回归 TTS 系统的质量，实验表明该方法显著提高了非自回归 TTS 系统的质量。