Although high-fidelity speech can be obtained for intralingual speech
synthesis, cross-lingual text-to-speech (CTTS) is still far from satisfactory
as it is difficult to accurately retain the speaker timbres(i.e. speaker
similarity) and eliminate the accents from their first language(i.e.
nativeness). In this paper, we demonstrated that vector-quantized(VQ) acoustic
feature contains less speaker information than mel-spectrogram. Based on this
finding, we propose a novel dual speaker embedding TTS (DSE-TTS) framework for
CTTS with authentic speaking style. Here, one embedding is fed to the acoustic
model to learn the linguistic speaking style, while the other one is integrated
into the vocoder to mimic the target speaker's timbre. Experiments show that by
combining both embeddings, DSE-TTS significantly outperforms the
state-of-the-art SANE-TTS in cross-lingual synthesis, especially in terms of
nativeness.

提出了一种新的双说话人嵌入式 TTS 框架，结合了一种嵌入式用于学习语言发音风格，另一个用于模拟目标说话人的音色，以解决跨语言 TTS 中的噪音干扰和说话人相似度问题。实验证明，这种框架在跨语言合成方面表现优异，尤其在语音自然度方面比最先进的方法要好。

DSE-TTS：用于跨语言文本转语音的双说话人嵌入

DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech

This paper presents a method for end-to-end cross-lingual text-to-speech
(TTS) which aims to preserve the target language's pronunciation regardless of
the original speaker's language. The model used is based on a non-attentive
Tacotron architecture, where the decoder has been replaced with a normalizing
flow network conditioned on the speaker identity, allowing both TTS and voice
conversion (VC) to be performed by the same model due to the inherent
linguistic content and speaker identity disentanglement. When used in a
cross-lingual setting, acoustic features are initially produced with a native
speaker of the target language and then voice conversion is applied by the same
model in order to convert these features to the target speaker's voice. We
verify through objective and subjective evaluations that our method can have
benefits compared to baseline cross-lingual synthesis. By including speakers
averaging 7.5 minutes of speech, we also present positive results on
low-resource scenarios.

该论文介绍了一种端到端的跨语言文字转语音方法，使用基于非注意力 Tacotron 架构的模型，并通过使用条件为说话人身份的归一化流网络，实现 TTS 和语音转换（VC）的可同时进行，该方法可以在低资源情景下获得良好效果。

基于流模型语音转换实现跨语言语音合成以提高发音

Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciation

Recent advances in cross-lingual text-to-speech (TTS) made it possible to
synthesize speech in a language foreign to a monolingual speaker. However,
there is still a large gap between the pronunciation of generated cross-lingual
speech and that of native speakers in terms of naturalness and intelligibility.
In this paper, a triplet training scheme is proposed to enhance the
cross-lingual pronunciation by allowing previously unseen content and speaker
combinations to be seen during training. Proposed method introduces an extra
fine-tune stage with triplet loss during training, which efficiently draws the
pronunciation of the synthesized foreign speech closer to those from the native
anchor speaker, while preserving the non-native speaker's timbre. Experiments
are conducted based on a state-of-the-art baseline cross-lingual TTS system and
its enhanced variants. All the objective and subjective evaluations show the
proposed method brings significant improvement in both intelligibility and
naturalness of the synthesized cross-lingual speech.

本文提出了一种三元组训练方案，借助额外的微调阶段和三元组损失在跨语言语音合成中提高发音自然度和可懂度。