We present a neural text-to-speech system for fine-grained prosody transfer
from one speaker to another. Conventional approaches for end-to-end prosody
transfer typically use either fixed-dimensional or variable-length prosody
embedding via a secondary attention to encode the reference signal. However,
when trained on a single-speaker dataset, the conventional prosody transfer
systems are not robust enough to speaker variability, especially in the case of
a reference signal coming from an unseen speaker. Therefore, we propose
decoupling of the reference signal alignment from the overall system. For this
purpose, we pre-compute phoneme-level time stamps and use them to aggregate
prosodic features per phoneme, injecting them into a sequence-to-sequence
text-to-speech system. We incorporate a variational auto-encoder to further
enhance the latent representation of prosody embeddings. We show that our
proposed approach is significantly more stable and achieves reliable prosody
transplantation from an unseen speaker. We also propose a solution to the use
case in which the transcription of the reference signal is absent. We evaluate
all our proposed methods using both objective and subjective listening tests.

本文提出一种神经文本转语音系统，通过使用变分自动编码器并在音标级别上聚合韵律特征，实现了从一个说话者到另一个说话者的精细韵律转移，并且解决了参考信号文本缺失的问题。主要关键字为 “神经文本转语音系统”、“韵律转移”、“变分自动编码器”、“音素级时间戳” 和 “序列到序列”。

面向单说者神经文本转语音的细粒度鲁棒性韵律转移

Fine-grained robust prosody transfer for single-speaker neural  text-to-speech

We present Deep Voice 3, a fully-convolutional attention-based neural
text-to-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural
speech synthesis systems in naturalness while training ten times faster. We
scale Deep Voice 3 to data set sizes unprecedented for TTS, training on more
than eight hundred hours of audio from over two thousand speakers. In addition,
we identify common error modes of attention-based speech synthesis networks,
demonstrate how to mitigate them, and compare several different waveform
synthesis methods. We also describe how to scale inference to ten million
queries per day on one single-GPU server.

Deep Voice 3 是一种全卷积自注意神经文本转语音系统，可以很好地匹配目前最先进的神经语音合成系统的自然度，同时训练速度快于十倍。通过在超过 2000 位演讲者的超过 800 小时的音频数据上进行训练，实现了 TTS 数据集规模的前所未有的扩展。同时，我们描述了如何缩放指向注意的语音合成网络的推断，以在单个 GPU 服务器上每天缩放到 1000 万个查询，并比较了几种不同的波形综合方法。