Multilingual Large Language Models (LLMs) have gained large popularity among
Natural Language Processing (NLP) researchers and practitioners. These models,
trained on huge datasets, show proficiency across various languages and
demonstrate effectiveness in numerous downstream tasks. This paper navigates
the landscape of multilingual LLMs, providing an introductory overview of their
technical aspects. It explains underlying architectures, objective functions,
pre-training data sources, and tokenization methods. This work explores the
unique features of different model types: encoder-only (mBERT, XLM-R),
decoder-only (XGLM, PALM, BLOOM, GPT-3), and encoder-decoder models (mT5,
mBART). Additionally, it addresses one of the significant limitations of
multilingual LLMs - the curse of multilinguality - and discusses current
attempts to overcome it.

多语言大型语言模型在自然语言处理领域的研究人员和从业者中广受欢迎。本文介绍了多语言大型语言模型的技术方面，并对其底层架构、目标函数、预训练数据来源和分词方法进行了概述。此外，还探讨了不同模型类型的独特特征，包括仅编码器模型、仅解码器模型和编码器 - 解码器模型。同时，还讨论了多语言大型语言模型的一个重要局限性，即多语言之间的相互影响问题，并探究了克服这一问题的当前尝试。