Instruction tuning has emerged as a powerful technique, significantly
boosting zero-shot performance on unseen tasks. While recent work has explored
cross-lingual generalization by applying instruction tuning to multilingual
models, previous studies have primarily focused on English, with a limited
exploration of non-English tasks. For an in-depth exploration of cross-lingual
generalization in instruction tuning, we perform instruction tuning
individually for two distinct language meta-datasets. Subsequently, we assess
the performance on unseen tasks in a language different from the one used for
training. To facilitate this investigation, we introduce a novel non-English
meta-dataset named "KORANI" (Korean Natural Instruction), comprising 51 Korean
benchmarks. Moreover, we design cross-lingual templates to mitigate
discrepancies in language and instruction-format of the template between
training and inference within the cross-lingual setting. Our experiments reveal
consistent improvements through cross-lingual generalization in both English
and Korean, outperforming baseline by average scores of 20.7\% and 13.6\%,
respectively. Remarkably, these enhancements are comparable to those achieved
by monolingual instruction tuning and even surpass them in some tasks. The
result underscores the significance of relevant data acquisition across
languages over linguistic congruence with unseen tasks during instruction
tuning.

通过跨语言调整指令和数据获取方法，研究表明对未知任务的指令调优在英文和韩文方面都取得了显著的改善效果，与单语指令调优相当甚至有些任务超过，强调了在指令调优过程中通过跨语言数据获取的相关性和语言一致性的重要性。