Approaches to improving multilingual language understanding often require
multiple languages during the training phase, rely on complicated training
techniques, and -- importantly -- struggle with significant performance gaps
between high-resource and low-resource languages. We hypothesize that the
performance gaps between languages are affected by linguistic gaps between
those languages and provide a novel solution for robust multilingual language
modeling by employing phonemic representations (specifically, using phonemes as
input tokens to LMs rather than subwords). We present quantitative evidence
from three cross-lingual tasks that demonstrate the effectiveness of phonemic
representation, which is further justified by a theoretical analysis of the
cross-lingual performance gap.

多语言理解的改进方法通常在训练阶段需要多种语言，依赖复杂的训练技术，同时也在高资源语言和低资源语言之间存在重大的性能差距。我们假设语言之间的性能差距受到这些语言之间的语言差距的影响，并通过使用声素表示（具体而言，使用声素作为输入标记到语言模型，而不是次词）来提供强大的多语种语言建模的新解决方案。我们通过三个跨语言任务的定量证据以及对跨语言性能差距的理论分析进一步证明了声素表示的有效性。