Recently, Large Language Models (LLMs) have shown impressive language
capabilities. However, most of the existing LLMs are all English-centric, which
have very unstable and unbalanced performance across different languages.
Multilingual alignment is an effective method to enhance the LLMs' multilingual
capabilities. In this work, we explore the multilingual alignment paradigm
which utilizes translation data and comprehensively investigate the spontaneous
multilingual improvement of LLMs. We find that LLMs only instruction-tuned on
question translation data without annotated answers are able to get significant
multilingual performance enhancement even across a wide range of languages
unseen during instruction-tuning. Additionally, we utilize different settings
and mechanistic interpretability methods to comprehensively analyze the LLM's
performance in the multilingual scenario.

通过多语言对齐方法，本文研究了大型语言模型的多语言能力提升，发现即使在没有注释答案的情况下，仅通过问题翻译数据进行训练的语言模型能够在广泛的未见过的语言中获得显著的性能提升，并利用不同的设置和机理解释方法对多语言场景下的语言模型性能进行了全面分析。