We present a multispeaker, multilingual text-to-speech (TTS) synthesis model
based on Tacotron that is able to produce high quality speech in multiple
languages. Moreover, the model is able to transfer voices across languages,
e.g. synthesize fluent Spanish speech using an English speaker's voice, without
training on any bilingual or parallel examples. Such transfer works across
distantly related languages, e.g. English and Mandarin.
Critical to achieving this result are: 1. using a phonemic input
representation to encourage sharing of model capacity across languages, and 2.
incorporating an adversarial loss term to encourage the model to disentangle
its representation of speaker identity (which is perfectly correlated with
language in the training data) from the speech content. Further scaling up the
model by training on multiple speakers of each language, and incorporating an
autoencoding input to help stabilize attention during training, results in a
model which can be used to consistently synthesize intelligible speech for
training speakers in all languages seen during training, and in native or
foreign accents.

该论文基于 Tacotron 合成模型，采用多说话人、多语言合成的方式，在没有任何双语或平行示例的情况下，实现了跨语言的语音转移。通过使用音素输入表征，鼓励模型跨语言共享模型容量，并结合对抗损失项，鼓励模型将语音内容与说话人身份表示分离，可以训练出一个可以用于所有训练语言的说话人的稳定可靠的语音合成模型。