In this paper, we propose a novel unsupervised text-to-speech (UTTS)
framework which does not require text-audio pairs for the TTS acoustic modeling
(AM). UTTS is a multi-speaker speech synthesizer that supports zero-shot voice
cloning, it is developed from a perspective of disentangled speech
representation learning. The framework offers a flexible choice of a speaker's
duration model, timbre feature (identity) and content for TTS inference. We
leverage recent advancements in self-supervised speech representation learning
as well as speech synthesis front-end techniques for system development.
Specifically, we employ our recently formulated Conditional Disentangled
Sequential Variational Auto-encoder (C-DSVAE) as the backbone UTTS AM, which
offers well-structured content representations given unsupervised alignment
(UA) as condition during training. For UTTS inference, we utilize a lexicon to
map input text to the phoneme sequence, which is expanded to the frame-level
forced alignment (FA) with a speaker-dependent duration model. Then, we develop
an alignment mapping module that converts FA to UA. Finally, the C-DSVAE,
serving as the self-supervised TTS AM, takes the predicted UA and a target
speaker embedding to generate the mel spectrogram, which is ultimately
converted to waveform with a neural vocoder. We show how our method enables
speech synthesis without using a paired TTS corpus. Experiments demonstrate
that UTTS can synthesize speech of high naturalness and intelligibility
measured by human and objective evaluations. Audio samples are available at our
demo page this https URL.

本文提出了一种新的无监督文本到语音（UTTS）框架，支持零样本语音克隆和高自然度、可理解度语音合成，是从分离的语音表示学习的角度开发的多说话人语音合成器，其利用了最近在自监督语音表示学习和语音合成前端技术方面的先进进展，在训练期间提供了条件分离的时序变分自编码器（C-DSVAE）作为骨干 UTTS AM。