We introduce an approach to multilingual speech synthesis which uses the
meta-learning concept of contextual parameter generation and produces
natural-sounding multilingual speech using more languages and less training
data than previous approaches. Our model is based on Tacotron 2 with a fully
convolutional input text encoder whose weights are predicted by a separate
parameter generator network. To boost voice cloning, the model uses an
adversarial speaker classifier with a gradient reversal layer that removes
speaker-specific information from the encoder.
We arranged two experiments to compare our model with baselines using various
levels of cross-lingual parameter sharing, in order to evaluate: (1) stability
and performance when training on low amounts of data, (2) pronunciation
accuracy and voice quality of code-switching synthesis. For training, we used
the CSS10 dataset and our new small dataset based on Common Voice recordings in
five languages. Our model is shown to effectively share information across
languages and according to a subjective evaluation test, it produces more
natural and accurate code-switching speech than the baselines.

本文介绍一种多语言语音合成方法，该方法使用上下文参数生成的元学习概念，使用更少的训练数据和更多的语言产生自然音质的多语言语音；其模型使用 Tacotron 2 和完全卷积输入文本编码器，在训练中使用 CSS10 数据集和新的基于 5 种语言的通用语音数据集，同时使用对抗分类器和梯度反转层增强语音克隆。实验结果表明，该模型跨语言信息共享效果良好，在 “代码切换” 语音合成中表现更自然、准确。

一个模型，多种语言：元学习应用于多语言文本朗读

One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech

We describe a neural network-based system for text-to-speech (TTS) synthesis
that is able to generate speech audio in the voice of many different speakers,
including those unseen during training. Our system consists of three
independently trained components: (1) a speaker encoder network, trained on a
speaker verification task using an independent dataset of noisy speech from
thousands of speakers without transcripts, to generate a fixed-dimensional
embedding vector from seconds of reference speech from a target speaker; (2) a
sequence-to-sequence synthesis network based on Tacotron 2, which generates a
mel spectrogram from text, conditioned on the speaker embedding; (3) an
auto-regressive WaveNet-based vocoder that converts the mel spectrogram into a
sequence of time domain waveform samples. We demonstrate that the proposed
model is able to transfer the knowledge of speaker variability learned by the
discriminatively-trained speaker encoder to the new task, and is able to
synthesize natural speech from speakers that were not seen during training. We
quantify the importance of training the speaker encoder on a large and diverse
speaker set in order to obtain the best generalization performance. Finally, we
show that randomly sampled speaker embeddings can be used to synthesize speech
in the voice of novel speakers dissimilar from those used in training,
indicating that the model has learned a high quality speaker representation.

描述了一个基于神经网络的文本转语音（TTS）合成系统，可以以许多不同讲话者的声音生成语音音频，该系统由三个独立训练的部分组成，包括训练说话者编码器网络进行讲话者验证任务，基于 Tacotron 2 的序列合成网络，以及将梅尔频谱图转换为时间领域波形样本的自回归 WaveNet 基声码器。

从说话人验证到多说话人语音合成的迁移学习

Transfer Learning from Speaker Verification to Multispeaker  Text-To-Speech Synthesis

This paper describes Tacotron 2, a neural network architecture for speech
synthesis directly from text. The system is composed of a recurrent
sequence-to-sequence feature prediction network that maps character embeddings
to mel-scale spectrograms, followed by a modified WaveNet model acting as a
vocoder to synthesize timedomain waveforms from those spectrograms. Our model
achieves a mean opinion score (MOS) of $4.53$ comparable to a MOS of $4.58$ for
professionally recorded speech. To validate our design choices, we present
ablation studies of key components of our system and evaluate the impact of
using mel spectrograms as the input to WaveNet instead of linguistic, duration,
and $F_0$ features. We further demonstrate that using a compact acoustic
intermediate representation enables significant simplification of the WaveNet
architecture.

该论文阐述了 Tacotron 2 的神经网络框架，该框架可以从文本中直接合成语音，其系统由一种递归的序列到序列的特征预测网络和一个修改的 WaveNet 模型组成，能够实现与专业录制的语音相当的平均意见分数 (MOS)。