Custom voice is to construct a personal speech synthesis system by adapting
the source speech synthesis model to the target model through the target few
recordings. The solution to constructing a custom voice is to combine an
adaptive acoustic model with a robust vocoder. However, training a robust
vocoder usually requires a multi-speaker dataset, which should include various
age groups and various timbres, so that the trained vocoder can be used for
unseen speakers. Collecting such a multi-speaker dataset is difficult, and the
dataset distribution always has a mismatch with the distribution of the target
speaker dataset. This paper proposes an adaptive vocoder for custom voice from
another novel perspective to solve the above problems. The adaptive vocoder
mainly uses a cross-domain consistency loss to solve the overfitting problem
encountered by the GAN-based neural vocoder in the transfer learning of
few-shot scenes. We construct two adaptive vocoders, AdaMelGAN and AdaHiFi-GAN.
First, We pre-train the source vocoder model on AISHELL3 and CSMSC datasets,
respectively. Then, fine-tune it on the internal dataset VXI-children with few
adaptation data. The empirical results show that a high-quality custom voice
system can be built by combining a adaptive acoustic model with a adaptive
vocoder.

本文通过交叉域一致性损失函数提出了适应性声码器算法解决了语音合成领域中将 GAN 进行少量样本的迁移学习时会存在的过拟合问题，同时在内部数据集上微调声码器模型，构建了两种自适应声码器 AdaMelGAN 和 AdaHiFi-GAN，该方法可用于构建高质量的自定义语音系统。

AdaVocoder：自适应定制语音编解码器

AdaVocoder: Adaptive Vocoder for Custom Voice

Custom voice, a specific text to speech (TTS) service in commercial speech
platforms, aims to adapt a source TTS model to synthesize personal voice for a
target speaker using few speech data. Custom voice presents two unique
challenges for TTS adaptation: 1) to support diverse customers, the adaptation
model needs to handle diverse acoustic conditions that could be very different
from source speech data, and 2) to support a large number of customers, the
adaptation parameters need to be small enough for each target speaker to reduce
memory usage while maintaining high voice quality. In this work, we propose
AdaSpeech, an adaptive TTS system for high-quality and efficient customization
of new voices. We design several techniques in AdaSpeech to address the two
challenges in custom voice: 1) To handle different acoustic conditions, we use
two acoustic encoders to extract an utterance-level vector and a sequence of
phoneme-level vectors from the target speech during training; in inference, we
extract the utterance-level vector from a reference speech and use an acoustic
predictor to predict the phoneme-level vectors. 2) To better trade off the
adaptation parameters and voice quality, we introduce conditional layer
normalization in the mel-spectrogram decoder of AdaSpeech, and fine-tune this
part in addition to speaker embedding for adaptation. We pre-train the source
TTS model on LibriTTS datasets and fine-tune it on VCTK and LJSpeech datasets
(with different acoustic conditions from LibriTTS) with few adaptation data,
e.g., 20 sentences, about 1 minute speech. Experiment results show that
AdaSpeech achieves much better adaptation quality than baseline methods, with
only about 5K specific parameters for each speaker, which demonstrates its
effectiveness for custom voice. Audio samples are available at
this https URL

提出了一个自适应的 TTS 系统 AdaSpeech，通过使用两个声学编码器处理不同的声学条件，并在 mel-spectrogram 解码器中引入条件层归一化技术来权衡自适应参数和语音质量，该系统在只有几分钟的训练数据的情况下，对个性化语音化的效果较好，适用于商业语音平台。