Custom voice, a specific text to speech (TTS) service in commercial speech
platforms, aims to adapt a source TTS model to synthesize personal voice for a
target speaker using few speech data. Custom voice presents two unique
challenges for TTS adaptation: 1) to support diverse customers, the adaptation
model needs to handle diverse acoustic conditions that could be very different
from source speech data, and 2) to support a large number of customers, the
adaptation parameters need to be small enough for each target speaker to reduce
memory usage while maintaining high voice quality. In this work, we propose
AdaSpeech, an adaptive TTS system for high-quality and efficient customization
of new voices. We design several techniques in AdaSpeech to address the two
challenges in custom voice: 1) To handle different acoustic conditions, we use
two acoustic encoders to extract an utterance-level vector and a sequence of
phoneme-level vectors from the target speech during training; in inference, we
extract the utterance-level vector from a reference speech and use an acoustic
predictor to predict the phoneme-level vectors. 2) To better trade off the
adaptation parameters and voice quality, we introduce conditional layer
normalization in the mel-spectrogram decoder of AdaSpeech, and fine-tune this
part in addition to speaker embedding for adaptation. We pre-train the source
TTS model on LibriTTS datasets and fine-tune it on VCTK and LJSpeech datasets
(with different acoustic conditions from LibriTTS) with few adaptation data,
e.g., 20 sentences, about 1 minute speech. Experiment results show that
AdaSpeech achieves much better adaptation quality than baseline methods, with
only about 5K specific parameters for each speaker, which demonstrates its
effectiveness for custom voice. Audio samples are available at
this https URL

提出了一个自适应的 TTS 系统 AdaSpeech，通过使用两个声学编码器处理不同的声学条件，并在 mel-spectrogram 解码器中引入条件层归一化技术来权衡自适应参数和语音质量，该系统在只有几分钟的训练数据的情况下，对个性化语音化的效果较好，适用于商业语音平台。