An essential design decision for multilingual Neural Text-To-Speech (NTTS)
systems is how to represent input linguistic features within the model. Looking
at the wide variety of approaches in the literature, two main paradigms emerge,
unified and separate representations. The former uses a shared set of phonetic
tokens across languages, whereas the latter uses unique phonetic tokens for
each language. In this paper, we conduct a comprehensive study comparing
multilingual NTTS systems models trained with both representations. Our results
reveal that the unified approach consistently achieves better cross-lingual
synthesis with respect to both naturalness and accent. Separate representations
tend to have an order of magnitude more tokens than unified ones, which may
affect model capacity. For this reason, we carry out an ablation study to
understand the interaction of the representation type with the size of the
token embedding. We find that the difference between the two paradigms only
emerges above a certain threshold embedding size. This study provides strong
evidence that unified representations should be the preferred paradigm when
building multilingual NTTS systems.

本研究比较了使用单一表示和分离表示的多语言神经文本语音合成系统，发现单一表示方法比分离表示在自然度和语音口音的交叉语言综合中表现一致更好。同时，单一表示方法具有更少的语音标记，从而提高了模型性能。