Multilingual Large Language Models (LLMs) have gained large popularity among
Natural Language Processing (NLP) researchers and practitioners. These models,
trained on huge datasets, show proficiency across various languages and
demonstrate effectiveness in numerous downstream tasks. This paper navigates
the landscape of multilingual LLMs, providing an introductory overview of their
technical aspects. It explains underlying architectures, objective functions,
pre-training data sources, and tokenization methods. This work explores the
unique features of different model types: encoder-only (mBERT, XLM-R),
decoder-only (XGLM, PALM, BLOOM, GPT-3), and encoder-decoder models (mT5,
mBART). Additionally, it addresses one of the significant limitations of
multilingual LLMs - the curse of multilinguality - and discusses current
attempts to overcome it.

多语言大型语言模型在自然语言处理领域的研究人员和从业者中广受欢迎。本文介绍了多语言大型语言模型的技术方面，并对其底层架构、目标函数、预训练数据来源和分词方法进行了概述。此外，还探讨了不同模型类型的独特特征，包括仅编码器模型、仅解码器模型和编码器 - 解码器模型。同时，还讨论了多语言大型语言模型的一个重要局限性，即多语言之间的相互影响问题，并探究了克服这一问题的当前尝试。

多语言大型语言模型与多语言性诅咒

Multilingual Large Language Models and Curse of Multilinguality

As for multilingual language models, it is important to select languages for
training because of the curse of multilinguality. (Conneau et al., 2020). It is
known that using languages with similar language structures is effective for
cross lingual transfer learning (Pires et al., 2019). However, we demonstrate
that using agglutinative languages such as Korean is more effective in cross
lingual transfer learning. This is a great discovery that will change the
training strategy of cross lingual transfer learning.

研究多语言语言模型的训练选择对于跨语言迁移学习至关重要，韩语等粘着语言的使用在跨语言迁移学习中效果更佳。这是一项重大发现，将改变跨语言迁移学习的训练策略。