While most research into speech synthesis has focused on synthesizing
high-quality speech for in-dataset speakers, an equally essential yet unsolved
problem is synthesizing speech for unseen speakers who are out-of-dataset with
limited reference data, i.e., speaker adaptive speech synthesis. Many studies
have proposed zero-shot speaker adaptive text-to-speech and voice conversion
approaches aimed at this task. However, most current approaches suffer from the
degradation of naturalness and speaker similarity when synthesizing speech for
unseen speakers (i.e., speakers not in the training dataset) due to the poor
generalizability of the model in out-of-distribution data. To address this
problem, we propose GZS-TV, a generalizable zero-shot speaker adaptive
text-to-speech and voice conversion model. GZS-TV introduces disentangled
representation learning for both speaker embedding extraction and timbre
transformation to improve model generalization and leverages the representation
learning capability of the variational autoencoder to enhance the speaker
encoder. Our experiments demonstrate that GZS-TV reduces performance
degradation on unseen speakers and outperforms all baseline models in multiple
datasets.

解决合成声音的自然性和说话人相似度降低的问题，我们提出了一种通用的零样本说话人自适应语音合成和声音转换模型。该模型使用解耦表示学习来改善模型的普适性，并利用变分自动编码器的表示学习能力增强说话人编码器，实验证明我们的模型在未知说话人上有更好的表现。