Personalizing a speech synthesis system is a highly desired application,
where the system can generate speech with the user's voice with rare enrolled
recordings. There are two main approaches to build such a system in recent
works: speaker adaptation and speaker encoding. On the one hand, speaker
adaptation methods fine-tune a trained multi-speaker text-to-speech (TTS) model
with few enrolled samples. However, they require at least thousands of
fine-tuning steps for high-quality adaptation, making it hard to apply on
devices. On the other hand, speaker encoding methods encode enrollment
utterances into a speaker embedding. The trained TTS model can synthesize the
user's speech conditioned on the corresponding speaker embedding. Nevertheless,
the speaker encoder suffers from the generalization gap between the seen and
unseen speakers.
In this paper, we propose applying a meta-learning algorithm to the speaker
adaptation method. More specifically, we use Model Agnostic Meta-Learning
(MAML) as the training algorithm of a multi-speaker TTS model, which aims to
find a great meta-initialization to adapt the model to any few-shot speaker
adaptation tasks quickly. Therefore, we can also adapt the meta-trained TTS
model to unseen speakers efficiently. Our experiments compare the proposed
method (Meta-TTS) with two baselines: a speaker adaptation method baseline and
a speaker encoding method baseline. The evaluation results show that Meta-TTS
can synthesize high speaker-similarity speech from few enrollment samples with
fewer adaptation steps than the speaker adaptation baseline and outperforms the
speaker encoding baseline under the same training scheme. When the speaker
encoder of the baseline is pre-trained with extra 8371 speakers of data,
Meta-TTS can still outperform the baseline on LibriTTS dataset and achieve
comparable results on VCTK dataset.

本文提出了一种使用元学习算法的多说话人文本转语音（Meta-TTS）模型，通过少量的适应步骤，可以生成具有高语音相似度的语音合成，并且比已有的说话人适应方法和说话人编码方法更为高效。

Meta-TTS: 面向 Few-Shot 说话人自适应的元学习文本转语音

Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech

Voice cloning is a highly desired feature for personalized speech interfaces.
Neural network based speech synthesis has been shown to generate high quality
speech for a large number of speakers. In this paper, we introduce a neural
voice cloning system that takes a few audio samples as input. We study two
approaches: speaker adaptation and speaker encoding. Speaker adaptation is
based on fine-tuning a multi-speaker generative model with a few cloning
samples. Speaker encoding is based on training a separate model to directly
infer a new speaker embedding from cloning audios and to be used with a
multi-speaker generative model. In terms of naturalness of the speech and its
similarity to original speaker, both approaches can achieve good performance,
even with very few cloning audios. While speaker adaptation can achieve better
naturalness and similarity, the cloning time or required memory for the speaker
encoding approach is significantly less, making it favorable for low-resource
deployment.

本文针对少量音频样本的情况，提出了一种基于神经网络的语音克隆系统，通过训练来进行说话人的自适应和编码，实现语音的自然性和相似度，适用于低资源系统的使用。