Most languages of the world pose low-resource challenges to natural language
processing models. With multilingual training, knowledge can be shared among
languages. However, not all languages positively influence each other and it is
an open research question how to select the most suitable set of languages for
multilingual training and avoid negative interference among languages whose
characteristics or data distributions are not compatible. In this paper, we
propose GradSim, a language grouping method based on gradient similarity. Our
experiments on three diverse multilingual benchmark datasets show that it leads
to the largest performance gains compared to other similarity measures and it
is better correlated with cross-lingual model performance. As a result, we set
the new state of the art on AfriSenti, a benchmark dataset for sentiment
analysis on low-resource African languages. In our extensive analysis, we
further reveal that besides linguistic features, the topics of the datasets
play an important role for language grouping and that lower layers of
transformer models encode language-specific features while higher layers
capture task-specific information.

通过 GradSim 方法进行语言分组，与其他相似性度量方法相比，在多个多语言基准数据集上取得最大的性能提升，与跨语言模型性能更好相关。此外，研究还发现除语言特征外，数据集的主题对语言分组也起着重要作用，而 Transformer 模型的较低层编码语言特定特征，而较高层捕捉任务特定信息。