In cross-lingual speech synthesis, the speech in various languages can be
synthesized for a monoglot speaker. Normally, only the data of monoglot
speakers are available for model training, thus the speaker similarity is
relatively low between the synthesized cross-lingual speech and the native
language recordings. Based on the multilingual transformer text-to-speech
model, this paper studies a multi-task learning framework to improve the
cross-lingual speaker similarity. To further improve the speaker similarity,
joint training with a speaker classifier is proposed. Here, a scheme similar to
parallel scheduled sampling is proposed to train the transformer model
efficiently to avoid breaking the parallel training mechanism when introducing
joint training. By using multi-task learning and speaker classifier joint
training, in subjective and objective evaluations, the cross-lingual speaker
similarity can be consistently improved for both the seen and unseen speakers
in the training set.

基于多语种转换器文本到语音模型，本文研究了一个多任务学习框架来提高跨语言的说话者相似度，并提出了与说话人分类器的联合训练来进一步提高说话者相似度。通过使用多任务学习和说话人分类器联合训练，该模型在主观和客观评估中，可以持续地提高训练集中已知和未知说话者的跨语言说话者相似度。