Multilingual proficiency presents a significant challenge for large language
models (LLMs). English-centric models are usually suboptimal in other
languages, particularly those that are linguistically distant from English.
This performance discrepancy mainly stems from the imbalanced distribution of
training data across languages during pre-training and instruction tuning
stages. To address this problem, we propose a novel approach called CrossIn,
which utilizes a mixed composition of cross-lingual instruction tuning data.
Our method leverages the compressed representation shared by various languages
to efficiently enhance the model's task-solving capabilities and multilingual
proficiency within a single process. In addition, we introduce a multi-task and
multi-faceted benchmark to evaluate the effectiveness of CrossIn. Experimental
results demonstrate that our method substantially improves performance across
tasks and languages, and we provide extensive insights into the impact of
cross-lingual data volume and the integration of translation data on enhancing
multilingual consistency and accuracy.

通过使用 CrossIn 方法，利用跨语言指导调整数据的混合构成，我们有效地提升了大语言模型在多语种任务和多语言能力上的表现，并对跨语言数据量和翻译数据的整合对提高多语言一致性和准确性的影响进行了广泛探讨。