In this work, we introduce a framework for cross-lingual speech synthesis,
which involves an upstream Voice Conversion (VC) model and a downstream
Text-To-Speech (TTS) model. The proposed framework consists of 4 stages. In the
first two stages, we use a VC model to convert utterances in the target locale
to the voice of the target speaker. In the third stage, the converted data is
combined with the linguistic features and durations from recordings in the
target language, which are then used to train a single-speaker acoustic model.
Finally, the last stage entails the training of a locale-independent vocoder.
Our evaluations show that the proposed paradigm outperforms state-of-the-art
approaches which are based on training a large multilingual TTS model. In
addition, our experiments demonstrate the robustness of our approach with
different model architectures, languages, speakers and amounts of data.
Moreover, our solution is especially beneficial in low-resource settings.

该研究提出了一个跨语言语音合成框架，包括上游声码转换模型和下游文本转语音模型。通过在四个阶段中使用声码转换模型将目标语音转换为目标说话人的声音、联合目标语言的语言特征和持续时间进行单一说话人声学模型的训练以及设计一种与地区无关的波形合成器等方法，我们的评估表明这种方法优于现有的基于多语言训练模型的方法，并在不同的模型结构、语言、说话人和数据量方面展现出其鲁棒性，尤其适用于资源匮乏环境。