The NLP research community has devoted increased attention to languages
beyond English, resulting in considerable improvements for multilingual NLP.
However, these improvements only apply to a small subset of the world's
languages. Aiming to extend this, an increasing number of papers aspires to
enhance generalizable multilingual performance across languages. To this end,
linguistic typology is commonly used to motivate language selection, on the
basis that a broad typological sample ought to imply generalization across a
broad range of languages. These selections are often described as being
'typologically diverse'. In this work, we systematically investigate NLP
research that includes claims regarding 'typological diversity'. We find there
are no set definitions or criteria for such claims. We introduce metrics to
approximate the diversity of language selection along several axes and find
that the results vary considerably across papers. Furthermore, we show that
skewed language selection can lead to overestimated multilingual performance.
We recommend future work to include an operationalization of 'typological
diversity' that empirically justifies the diversity of language samples.

本研究系统地调查了涉及 “类型多样性” 的自然语言处理研究，发现并无固定的定义或标准。我们引入了几种度量标准来近似衡量语言选择的多样性，发现结果在各篇论文中差异很大。此外，我们证明了偏斜的语言选择可能导致对多语言性能的高估。我们建议未来的研究应当在 “类型多样性” 方面提供清晰的操作定义，以从经验证据上证明语言样本的多样性。

自然语言处理中的 ' 类型多样性 ' 是什么？

What is 'Typological Diversity' in NLP?

Cross-lingual transfer, where a high-resource transfer language is used to
improve the accuracy of a low-resource task language, is now an invaluable tool
for improving performance of natural language processing (NLP) on low-resource
languages. However, given a particular task language, it is not clear which
language to transfer from, and the standard strategy is to select languages
based on ad hoc criteria, usually the intuition of the experimenter. Since a
large number of features contribute to the success of cross-lingual transfer
(including phylogenetic similarity, typological properties, lexical overlap, or
size of available data), even the most enlightened experimenter rarely
considers all these factors for the particular task at hand. In this paper, we
consider this task of automatically selecting optimal transfer languages as a
ranking problem, and build models that consider the aforementioned features to
perform this prediction. In experiments on representative NLP tasks, we
demonstrate that our model predicts good transfer languages much better than ad
hoc baselines considering single features in isolation, and glean insights on
what features are most informative for each different NLP tasks, which may
inform future ad hoc selection even without use of our method. Code, data, and
pre-trained models are available at this https URL

该论文探讨了如何自动选择最佳转移语言作为排名问题，建立模型考虑诸如语言相似性，类型学特性，词汇重叠或可用数据大小等因素以执行这种预测。在代表性 NLP 任务的实验中，我们证明了我们的模型比单一特征的单独考虑的基准线更好地预测良好的转移语言，并了解了哪些特征对每种不同的 NLP 任务是最有信息量的，这可能会为未来的专家选择提供帮助。