Modelling prosody variation is critical for synthesizing natural and
expressive speech in end-to-end text-to-speech (TTS) systems. In this paper, a
cross-utterance conditional VAE (CUC-VAE) is proposed to estimate a posterior
probability distribution of the latent prosody features for each phoneme by
conditioning on acoustic features, speaker information, and text features
obtained from both past and future sentences. At inference time, instead of the
standard Gaussian distribution used by VAE, CUC-VAE allows sampling from an
utterance-specific prior distribution conditioned on cross-utterance
information, which allows the prosody features generated by the TTS system to
be related to the context and is more similar to how humans naturally produce
prosody. The performance of CUC-VAE is evaluated via a qualitative listening
test for naturalness, intelligibility and quantitative measurements, including
word error rates and the standard deviation of prosody attributes. Experimental
results on LJ-Speech and LibriTTS data show that the proposed CUC-VAE TTS
system improves naturalness and prosody diversity with clear margins.

本文提出了一种跨话语 CUC-VAE 模型来估计每个音素的潜在韵律特征的后验概率分布，该模型结合语音，说话人信息和文本特征，并允许与上下文相关的生成韵律特征，实验结果表明此模型可以显著提高其自然度和韵律分布。