We explore the impact of leveraging the relatedness of languages that belong
to the same family in NLP models using multilingual fine-tuning. We hypothesize
and validate that multilingual fine-tuning of pre-trained language models can
yield better performance on downstream NLP applications, compared to models
fine-tuned on individual languages. A first of its kind detailed study is
presented to track performance change as languages are added to a base language
in a graded and greedy (in the sense of best boost of performance) manner;
which reveals that careful selection of subset of related languages can
significantly improve performance than utilizing all related languages. The
Indo-Aryan (IA) language family is chosen for the study, the exact languages
being Bengali, Gujarati, Hindi, Marathi, Oriya, Punjabi and Urdu. The script
barrier is crossed by simple rule-based transliteration of the text of all
languages to Devanagari. Experiments are performed on mBERT, IndicBERT, MuRIL
and two RoBERTa-based LMs, the last two being pre-trained by us. Low resource
languages, such as Oriya and Punjabi, are found to be the largest beneficiaries
of multilingual fine-tuning. Textual Entailment, Entity Classification, Section
Title Prediction, tasks of IndicGLUE and POS tagging form our test bed.
Compared to monolingual fine tuning we get relative performance improvement of
up to 150% in the downstream tasks. The surprise take-away is that for any
language there is a particular combination of other languages which yields the
best performance, and any additional language is in fact detrimental.

本文探讨利用多语言微调技术在 NLP 模型中利用同一语系的相关性对性能的影响，结果表明在微调预训练的多语言模型时，与单独微调每一语言相比，多语言微调可以在下游 NLP 应用程序中获得更好的性能。 印欧 - 雅利安语系（IA）被选为研究对象，实验结果表明精细选择一组相关语言的性能比使用所有相关语言显着提高。

语言相关性在多语言语言模型微调中的作用：印度雅利安语的案例研究

Role of Language Relatedness in Multilingual Fine-tuning of Language  Models: A Case Study in Indo-Aryan Languages

Lexical ambiguity can impede NLP systems from accurate understanding of
semantics. Despite its potential benefits, the integration of sense-level
information into NLP systems has remained understudied. By incorporating a
novel disambiguation algorithm into a state-of-the-art classification model, we
create a pipeline to integrate sense-level information into downstream NLP
applications. We show that a simple disambiguation of the input text can lead
to consistent performance improvement on multiple topic categorization and
polarity detection datasets, particularly when the fine granularity of the
underlying sense inventory is reduced and the document is sufficiently large.
Our results also point to the need for sense representation research to focus
more on in vivo evaluations which target the performance in downstream NLP
applications rather than artificial benchmarks.

通过将新颖的消歧算法与最先进的分类模型相结合，我们创建了一个流程来将语义级别的信息整合进下游自然语言处理应用程序中，尤其是当底层语义清单的细粒度降低且文档足够大时，在多个主题分类和极性检测数据集上，我们发现对输入文本进行简单的消歧可以带来一致的性能提升。我们的结果还指出，需要更多地关注关注语义表示研究中基于下游自然语言处理应用程序的实际评估，而不是人造基准。