Multilingual Language Models offer a way to incorporate multiple languages in
one model and utilize cross-language transfer learning to improve performance
for different Natural Language Processing (NLP) tasks. Despite progress in
multilingual models, not all languages are supported as well, particularly in
low-resource settings. In this work, we investigate the linguistic
representation of different languages in multilingual models. We start by
asking the question which languages are supported in popular multilingual
models and which languages are left behind. Then, for included languages, we
look at models' learned representations based on language family and dialect
and try to understand how models' learned representations for~(1) seen and~(2)
unseen languages vary across different language groups. In addition, we test
and analyze performance on downstream tasks such as text generation and Named
Entity Recognition. We observe from our experiments that community-centered
models -- models that focus on languages of a given family or geographical
location and are built by communities who speak them -- perform better at
distinguishing between languages in the same family for low-resource languages.
Our paper contributes to the literature in understanding multilingual models
and their shortcomings and offers insights on potential ways to improve them.

通过研究多语言模型的语言学表示，我们发现针对低资源语言，以某个语言家族或地理位置为重点且由这些语言使用者构建的社区中心模型在区分同一语言家族的语言方面表现更好，有助于理解多语言模型的问题并提供改进方法。