The successful adaptation of multilingual language models (LMs) to a specific
language-task pair critically depends on the availability of data tailored for
that condition. While cross-lingual transfer (XLT) methods have contributed to
addressing this data scarcity problem, there still exists ongoing debate about
the mechanisms behind their effectiveness. In this work, we focus on one of
promising assumptions about inner workings of XLT, that it encourages
multilingual LMs to place greater emphasis on language-agnostic or
task-specific features. We test this hypothesis by examining how the patterns
of XLT change with a varying number of source languages involved in the
process. Our experimental findings show that the use of multiple source
languages in XLT-a technique we term Multi-Source Language Training
(MSLT)-leads to increased mingling of embedding spaces for different languages,
supporting the claim that XLT benefits from making use of language-independent
information. On the other hand, we discover that using an arbitrary combination
of source languages does not always guarantee better performance. We suggest
simple heuristics for identifying effective language combinations for MSLT and
empirically prove its effectiveness.

多语种语言模型，交叉语言迁移，源语言，嵌入空间，语言组合

跨语言迁移中多源语言训练的分析

Analysis of Multi-Source Language Training in Cross-Lingual Transfer

Cross-lingual transfer (XLT) is an emergent ability of multilingual language
models that preserves their performance on a task to a significant extent when
evaluated in languages that were not included in the fine-tuning process. While
English, due to its widespread usage, is typically regarded as the primary
language for model adaption in various tasks, recent studies have revealed that
the efficacy of XLT can be amplified by selecting the most appropriate source
languages based on specific conditions. In this work, we propose the
utilization of sub-network similarity between two languages as a proxy for
predicting the compatibility of the languages in the context of XLT. Our
approach is model-oriented, better reflecting the inner workings of foundation
models. In addition, it requires only a moderate amount of raw text from
candidate languages, distinguishing it from the majority of previous methods
that rely on external resources. In experiments, we demonstrate that our method
is more effective than baselines across diverse tasks. Specifically, it shows
proficiency in ranking candidates for zero-shot XLT, achieving an improvement
of 4.6% on average in terms of NDCG@3. We also provide extensive analyses that
confirm the utility of sub-networks for XLT prediction.

通过使用两种语言之间的子网络相似性作为预测跨语言转移 (XLT) 的语言兼容性的代理方法，本研究提出了一种以模型为导向，仅需要适量原始文本的方法，进一步提高了 XLT 的效能。实验证明我们的方法在多样性任务上优于其他基准模型，并能有效对零样本 XLT 进行排名，平均 NDCG@3 提高了 4.6%。此外，我们还对子网络在 XLT 预测中的实用性进行了广泛的分析。

X-SNS：通过子网络相似性进行跨语言转移预测

X-SNS: Cross-Lingual Transfer Prediction through Sub-Network Similarity

We tackle the task of automatically discriminating between human and machine
translations. As opposed to most previous work, we perform experiments in a
multilingual setting, considering multiple languages and multilingual
pretrained language models. We show that a classifier trained on parallel data
with a single source language (in our case German-English) can still perform
well on English translations that come from different source languages, even
when the machine translations were produced by other systems than the one it
was trained on. Additionally, we demonstrate that incorporating the source text
in the input of a multilingual classifier improves (i) its accuracy and (ii)
its robustness on cross-system evaluation, compared to a monolingual
classifier. Furthermore, we find that using training data from multiple source
languages (German, Russian, and Chinese) tends to improve the accuracy of both
monolingual and multilingual classifiers. Finally, we show that bilingual
classifiers and classifiers trained on multiple source languages benefit from
being trained on longer text sequences, rather than on sentences.

研究人员利用多语言数据和预训练的语言模型，成功开发出一种分类器，能够自动区分不同源语言翻译的人工翻译和机器翻译，发现使用少量数据训练的分类器的推广能力更好。