This paper aims to enhance low-resource TTS by reducing training data
requirements using compact speech representations. A Multi-Stage Multi-Codebook
(MSMC) VQ-GAN is trained to learn the representation, MSMCR, and decode it to
waveforms. Subsequently, we train the multi-stage predictor to predict MSMCRs
from the text for TTS synthesis. Moreover, we optimize the training strategy by
leveraging more audio to learn MSMCRs better for low-resource languages. It
selects audio from other languages using speaker similarity metric to augment
the training set, and applies transfer learning to improve training quality. In
MOS tests, the proposed system significantly outperforms FastSpeech and VITS in
standard and low-resource scenarios, showing lower data requirements. The
proposed training strategy effectively enhances MSMCRs on waveform
reconstruction. It improves TTS performance further, which wins 77% votes in
the preference test for the low-resource TTS with only 15 minutes of paired
data.

本文提出了一种用于提高低资源的 TTS 系统性能的方法，其利用紧凑的语音表示并利用 Multi-Stage Multi-Codebook (MSMC) VQ-GAN 学习 MSMCR 表示并解码成波形，并使用多阶段预测器从文本中预测 MSMCRs 进行 TTS 合成，并通过优化训练策略，利用训练集增强了微型语言环境下的训练质量，并在 MOS 测试中显示出优异的性能。

学习紧凑的语音表示以实现低资源语言的高质量神经语音合成

Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representations

State-of-the-art text-to-speech (TTS) systems require several hours of
recorded speech data to generate high-quality synthetic speech. When using
reduced amounts of training data, standard TTS models suffer from speech
quality and intelligibility degradations, making training low-resource TTS
systems problematic. In this paper, we propose a novel extremely low-resource
TTS method called Voice Filter that uses as little as one minute of speech from
a target speaker. It uses voice conversion (VC) as a post-processing module
appended to a pre-existing high-quality TTS system and marks a conceptual shift
in the existing TTS paradigm, framing the few-shot TTS problem as a VC task.
Furthermore, we propose to use a duration-controllable TTS system to create a
parallel speech corpus to facilitate the VC task. Results show that the Voice
Filter outperforms state-of-the-art few-shot speech synthesis techniques in
terms of objective and subjective metrics on one minute of speech on a diverse
set of voices, while being competitive against a TTS model built on 30 times
more data.

本文提出一种极低资源语音合成方法 Voice Filter，它只需要使用目标发言人的一分钟语音，采用基于声音转换（VC）的后处理模块来增强现有的高质量 TTS 系统，成功地解决了低资源 TTS 系统训练中的质量和可理解性降低的问题，并将少样本 TTS 问题视为 VC 任务。此外，本文还提出使用一种持续可控的 TTS 系统创建平行语音语料库以促进 VC 任务。结果表明，Voice Filter 在多个基于大量语音数据声音的客观和主观度量上表现优异。