While recent text-to-speech (TTS) systems have made remarkable strides toward
human-level quality, the performance of cross-lingual TTS lags behind that of
intra-lingual TTS. This gap is mainly rooted from the speaker-language
entanglement problem in cross-lingual TTS. In this paper, we propose
CrossSpeech which improves the quality of cross-lingual speech by effectively
disentangling speaker and language information in the level of acoustic feature
space. Specifically, CrossSpeech decomposes the speech generation pipeline into
the speaker-independent generator (SIG) and speaker-dependent generator (SDG).
The SIG produces the speaker-independent acoustic representation which is not
biased to specific speaker distributions. On the other hand, the SDG models
speaker-dependent speech variation that characterizes speaker attributes. By
handling each information separately, CrossSpeech can obtain disentangled
speaker and language representations. From the experiments, we verify that
CrossSpeech achieves significant improvements in cross-lingual TTS, especially
in terms of speaker similarity to the target speaker.

本文提出 CrossSpeech，通过在声学特征级别上有效地分离说话人和语言信息来提高跨语言语音的质量。具体而言，CrossSpeech 将语音生成流程分解为 SIG 和 SDG，通过分别处理每个信息，可以获得分离的说话人和语言表示。通过实验证明，CrossSpeech 在跨语言 TTS 中取得了显著的改进，特别是在与目标说话人的说话人相似性方面。

CrossSpeech: 跨语种语音合成的发音无关音频表征

CrossSpeech: Speaker-independent Acoustic Representation for Cross-lingual Speech Synthesis

This paper studies a transferable phoneme embedding framework that aims to
deal with the cross-lingual text-to-speech (TTS) problem under the few-shot
setting. Transfer learning is a common approach when it comes to few-shot
learning since training from scratch on few-shot training data is bound to
overfit. Still, we find that the naive transfer learning approach fails to
adapt to unseen languages under extremely few-shot settings, where less than 8
minutes of data is provided. We deal with the problem by proposing a framework
that consists of a phoneme-based TTS model and a codebook module to project
phonemes from different languages into a learned latent space. Furthermore, by
utilizing phoneme-level averaged self-supervised learned features, we
effectively improve the quality of synthesized speeches. Experiments show that
using 4 utterances, which is about 30 seconds of data, is enough to synthesize
intelligible speech when adapting to an unseen language using our framework.

本文研究一种可转移的音位嵌入框架，旨在解决少样本跨语言语音合成问题。在 8 分钟以下数据提供的极少样本情况下，作者提出了包括基于音位 TTS 模型和码本模块在内的框架。通过利用音位级平均的自我监督学习特征，实现了合成语音质量的有效改进。实验证明，使用 30 秒左右数据就能够合成出可理解的说话声音。