Although high-fidelity speech can be obtained for intralingual speech
synthesis, cross-lingual text-to-speech (CTTS) is still far from satisfactory
as it is difficult to accurately retain the speaker timbres(i.e. speaker
similarity) and eliminate the accents from their first language(i.e.
nativeness). In this paper, we demonstrated that vector-quantized(VQ) acoustic
feature contains less speaker information than mel-spectrogram. Based on this
finding, we propose a novel dual speaker embedding TTS (DSE-TTS) framework for
CTTS with authentic speaking style. Here, one embedding is fed to the acoustic
model to learn the linguistic speaking style, while the other one is integrated
into the vocoder to mimic the target speaker's timbre. Experiments show that by
combining both embeddings, DSE-TTS significantly outperforms the
state-of-the-art SANE-TTS in cross-lingual synthesis, especially in terms of
nativeness.

提出了一种新的双说话人嵌入式 TTS 框架，结合了一种嵌入式用于学习语言发音风格，另一个用于模拟目标说话人的音色，以解决跨语言 TTS 中的噪音干扰和说话人相似度问题。实验证明，这种框架在跨语言合成方面表现优异，尤其在语音自然度方面比最先进的方法要好。