Large language models (LLMs) have demonstrated remarkable prowess in language
understanding and generation. Advancing from foundation LLMs to
instructionfollowing LLMs, instruction tuning plays a vital role in aligning
LLMs to human preferences. However, the existing LLMs are usually focused on
English, leading to inferior performance in non-English languages. In order to
improve the performance for non-English languages, it is necessary to collect
language-specific training data for foundation LLMs and construct
language-specific instructions for instruction tuning, both of which are heavy
loads. To minimize human workload, we propose to transfer the capabilities of
language generation and instruction following from English to other languages
through an interactive translation task. We have developed BayLing, an
instruction-following LLM by utilizing LLaMA as the foundation LLM and
automatically constructing interactive translation instructions for instructing
tuning. Extensive assessments demonstrate that BayLing achieves comparable
performance to GPT-3.5-turbo, despite utilizing a considerably smaller
parameter size of only 13 billion. Experimental results on translation tasks
show that BayLing achieves 95% of single-turn translation capability compared
to GPT-4 with automatic evaluation and 96% of interactive translation
capability compared to GPT-3.5-turbo with human evaluation. To estimate the
performance on general tasks, we created a multi-turn instruction test set
called BayLing-80. The experimental results on BayLing-80 indicate that BayLing
achieves 89% of performance compared to GPT-3.5-turbo. BayLing also
demonstrates outstanding performance on knowledge assessment of Chinese GaoKao
and English SAT, second only to GPT-3.5-turbo among a multitude of
instruction-following LLMs. Demo, homepage, code and models of BayLing are
available.

这篇文章提出了一种新的方法，通过互动翻译任务将英语的语言生成和指令遵循能力转移到其他语言，从而构建面向非英语语言的语言模型 BayLing。实验结果表明 BayLing 在词语翻译和多轮指令跟进等任务上都表现出了优异的性能。