Modern NLP breakthrough includes large multilingual models capable of
performing tasks across more than 100 languages. State-of-the-art language
models came a long way, starting from the simple one-hot representation of
words capable of performing tasks like natural language understanding,
common-sense reasoning, or question-answering, thus capturing both the syntax
and semantics of texts. At the same time, language models are expanding beyond
our known language boundary, even competitively performing over very
low-resource dialects of endangered languages. However, there are still
problems to solve to ensure an equitable representation of texts through a
unified modeling space across language and speakers. In this survey, we shed
light on this iterative progression of multilingual text representation and
discuss the driving factors that ultimately led to the current
state-of-the-art. Subsequently, we discuss how the full potential of language
democratization could be obtained, reaching beyond the known limits and what is
the scope of improvement in that space.

现代自然语言处理的突破包括能够在 100 多种语言中执行任务的大型多语种模型。最先进的语言模型从简单的独热编码词表示开始，能够执行自然语言理解、常识推理或问答等任务，从而捕捉文本的语法和语义。同时，语言模型正在超越我们所知的语言界限，甚至在资源非常有限的濒危语言的方言中获得有竞争力的表现。然而，仍然存在一些问题，需要解决，以确保通过统一的建模空间在语言和说话者之间获得公正的文本表示。在本调研中，我们阐述了多语种文本表示的迭代进展，并讨论了最终导致当前最先进技术的推动因素。随后，我们讨论了如何实现语言民主化的全部潜力，突破所知的限制，并探讨在该领域的改进范围。