The language ability of Large Language Models (LLMs) is often unbalanced
towards English because of the imbalance in the distribution of the
pre-training data. This disparity is demanded in further fine-tuning and
affecting the cross-lingual abilities of LLMs. In this paper, we propose to
empower Instructiontuned LLMs (It-LLMs) in languages other than English by
building semantic alignment between them. Hence, we propose CrossAlpaca, an
It-LLM with cross-lingual instruction-following and Translation-following
demonstrations to improve semantic alignment between languages. We validate our
approach on the multilingual Question Answering (QA) benchmarks XQUAD and MLQA
and adapted versions of MMLU and BBH. Our models, tested over six different
languages, outperform the It-LLMs tuned on monolingual data. The final results
show that instruction tuning on non-English data is not enough and that
semantic alignment can be further improved by Translation-following
demonstrations.

通过在语言模型中建立语义对齐，该研究提出了一种在英语以外的语言中增强指令调整型大型语言模型（It-LLMs）的方法，通过交叉语言指导和翻译指导演示，提高语义对齐，并在六种不同语言上的多语言问答基准测试中验证其方法的效果。