Instruction tuning has remarkably advanced large language models (LLMs) in
understanding and responding to diverse human instructions. Despite the success
in high-resource languages, its application in lower-resource ones faces
challenges due to the imbalanced foundational abilities of LLMs across
different languages, stemming from the uneven language distribution in their
pre-training data. To tackle this issue, we propose pivot language guided
generation (PLUG), an approach that utilizes a high-resource language,
primarily English, as the pivot to enhance instruction tuning in lower-resource
languages. It trains the model to first process instructions in the pivot
language, and then produce responses in the target language. To evaluate our
approach, we introduce a benchmark, X-AlpacaEval, of instructions in 4
languages (Chinese, Korean, Italian, and Spanish), each annotated by
professional translators. Our approach demonstrates a significant improvement
in the instruction-following abilities of LLMs by 29% on average, compared to
directly responding in the target language alone. Further experiments validate
the versatility of our approach by employing alternative pivot languages beyond
English to assist languages where LLMs exhibit lower proficiency.

使用英语作为中心语言，我们提出了一种名为 PLUG 的方法来增强低资源语言中指令调优的能力，通过首先在中心语言处理指令，然后在目标语言进行回应，我们的方法相较于仅在目标语言回应指令的能力提高了 29%。