Multilingual Large Language Models (LLMs) have gained large popularity among
Natural Language Processing (NLP) researchers and practitioners. These models,
trained on huge datasets, show proficiency across various languages and
demonstrate effectiveness in numerous downstream tasks. This paper navigates
the landscape of multilingual LLMs, providing an introductory overview of their
technical aspects. It explains underlying architectures, objective functions,
pre-training data sources, and tokenization methods. This work explores the
unique features of different model types: encoder-only (mBERT, XLM-R),
decoder-only (XGLM, PALM, BLOOM, GPT-3), and encoder-decoder models (mT5,
mBART). Additionally, it addresses one of the significant limitations of
multilingual LLMs - the curse of multilinguality - and discusses current
attempts to overcome it.

多语言大型语言模型在自然语言处理领域的研究人员和从业者中广受欢迎。本文介绍了多语言大型语言模型的技术方面，并对其底层架构、目标函数、预训练数据来源和分词方法进行了概述。此外，还探讨了不同模型类型的独特特征，包括仅编码器模型、仅解码器模型和编码器 - 解码器模型。同时，还讨论了多语言大型语言模型的一个重要局限性，即多语言之间的相互影响问题，并探究了克服这一问题的当前尝试。

多语言大型语言模型与多语言性诅咒

Multilingual Large Language Models and Curse of Multilinguality

This paper looks at philosophical questions that arise in the context of AI
alignment. It defends three propositions. First, normative and technical
aspects of the AI alignment problem are interrelated, creating space for
productive engagement between people working in both domains. Second, it is
important to be clear about the goal of alignment. There are significant
differences between AI that aligns with instructions, intentions, revealed
preferences, ideal preferences, interests and values. A principle-based
approach to AI alignment, which combines these elements in a systematic way,
has considerable advantages in this context. Third, the central challenge for
theorists is not to identify 'true' moral principles for AI; rather, it is to
identify fair principles for alignment, that receive reflective endorsement
despite widespread variation in people's moral beliefs. The final part of the
paper explores three ways in which fair principles for AI alignment could
potentially be identified.

探讨人工智能对齐问题所涉及的哲学问题，明确目标并提出合理的原则方法来解决技术和规范的问题，从而达成公平的人工智能对齐。