This paper proposes VARA-TTS, a non-autoregressive (non-AR) text-to-speech (TTS) model using a very deep Variational Autoencoder (VDVAE) with Residual Attention mechanism, which refines the textual-to-acoustic alignment layer-wisely. Hierarchical latent variables with different temporal resolutions from the VDVAE are used as queries for residual attention module. By leveraging the coarse global alignment from previous attention layer as an extra input, the following attention layer can produce a refined version of alignment. This amortizes the burden of learning the textual-to-acoustic alignment among multiple attention layers and outperforms the use of only a single attention layer in robustness. An utterance-level speaking speed factor is computed by a jointly-trained speaking speed predictor, which takes the mean-pooled latent variables of the coarsest layer as input, to determine number of acoustic frames at inference. Experimental results show that VARA-TTS achieves slightly inferior speech quality to an AR counterpart Tacotron 2 but an order-of-magnitude speed-up at inference; and outperforms an analogous non-AR model, BVAE-TTS, in terms of speech quality.

本文提出了一个采用多层变分自动编码器和残差注意机制的VARA-TTS非自回归文本到语音模型，通过逐层精细化文本到声学对齐，将学习负担分摊到多个注意力层中，相比单个注意力层具有更强的稳健性，并使用一个共同训练的发音速度预测器计算话语级别的发音速度，以确定推理时的声学帧数。实验结果表明，VARA-TTS在推理速度上具有数量级的提升，并且语音质量略逊于AR模型Tacotron 2，但在语音质量方面优于相似的非自回归模型BVAE-TTS。

基于非自回归式、深度VAE和残差注意力的文本转语音合成算法VARA-TTS