In this work, we introduce a framework for cross-lingual speech synthesis,
which involves an upstream Voice Conversion (VC) model and a downstream
Text-To-Speech (TTS) model. The proposed framework consists of 4 stages. In the
first two stages, we use a VC model to convert utterances in the target locale
to the voice of the target speaker. In the third stage, the converted data is
combined with the linguistic features and durations from recordings in the
target language, which are then used to train a single-speaker acoustic model.
Finally, the last stage entails the training of a locale-independent vocoder.
Our evaluations show that the proposed paradigm outperforms state-of-the-art
approaches which are based on training a large multilingual TTS model. In
addition, our experiments demonstrate the robustness of our approach with
different model architectures, languages, speakers and amounts of data.
Moreover, our solution is especially beneficial in low-resource settings.

该研究提出了一个跨语言语音合成框架，包括上游声码转换模型和下游文本转语音模型。通过在四个阶段中使用声码转换模型将目标语音转换为目标说话人的声音、联合目标语言的语言特征和持续时间进行单一说话人声学模型的训练以及设计一种与地区无关的波形合成器等方法，我们的评估表明这种方法优于现有的基于多语言训练模型的方法，并在不同的模型结构、语言、说话人和数据量方面展现出其鲁棒性，尤其适用于资源匮乏环境。

通过基于流的语音转换进行跨语言知识蒸馏，用于稳健的多语言文本到语音转换

Cross-lingual Knowledge Distillation via Flow-based Voice Conversion for  Robust Polyglot Text-To-Speech

In cross-lingual speech synthesis, the speech in various languages can be
synthesized for a monoglot speaker. Normally, only the data of monoglot
speakers are available for model training, thus the speaker similarity is
relatively low between the synthesized cross-lingual speech and the native
language recordings. Based on the multilingual transformer text-to-speech
model, this paper studies a multi-task learning framework to improve the
cross-lingual speaker similarity. To further improve the speaker similarity,
joint training with a speaker classifier is proposed. Here, a scheme similar to
parallel scheduled sampling is proposed to train the transformer model
efficiently to avoid breaking the parallel training mechanism when introducing
joint training. By using multi-task learning and speaker classifier joint
training, in subjective and objective evaluations, the cross-lingual speaker
similarity can be consistently improved for both the seen and unseen speakers
in the training set.

基于多语种转换器文本到语音模型，本文研究了一个多任务学习框架来提高跨语言的说话者相似度，并提出了与说话人分类器的联合训练来进一步提高说话者相似度。通过使用多任务学习和说话人分类器联合训练，该模型在主观和客观评估中，可以持续地提高训练集中已知和未知说话者的跨语言说话者相似度。