Domain Adaptation (DA) of Neural Machine Translation (NMT) model often relies
on a pre-trained general NMT model which is adapted to the new domain on a
sample of in-domain parallel data. Without parallel data, there is no way to
estimate the potential benefit of DA, nor the amount of parallel samples it
would require. It is however a desirable functionality that could help MT
practitioners to make an informed decision before investing resources in
dataset creation. We propose a Domain adaptation Learning Curve prediction
(DaLC) model that predicts prospective DA performance based on in-domain
monolingual samples in the source language. Our model relies on the NMT encoder
representations combined with various instance and corpus-level features. We
demonstrate that instance-level is better able to distinguish between different
domains compared to corpus-level frameworks proposed in previous studies.
Finally, we perform in-depth analyses of the results highlighting the
limitations of our approach, and provide directions for future research.

本文提出了一种基于源语种单语数据的领域适应学习曲线预测模型，该模型利用 NMT 编码器表示与各种实例和语料库级别特征相结合，能更好地区分不同领域，并探讨了该方法的局限性和未来研究方向。

DaLC: 神经机器翻译域适应学习曲线预测

DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation

Multilingual Neural Machine Translation (NMT) models have yielded large
empirical success in transfer learning settings. However, these black-box
representations are poorly understood, and their mode of transfer remains
elusive. In this work, we attempt to understand massively multilingual NMT
representations (with 103 languages) using Singular Value Canonical Correlation
Analysis (SVCCA), a representation similarity framework that allows us to
compare representations across different languages, layers and models. Our
analysis validates several empirical results and long-standing intuitions, and
unveils new observations regarding how representations evolve in a multilingual
translation model. We draw three major conclusions from our analysis, with
implications on cross-lingual transfer learning: (i) Encoder representations of
different languages cluster based on linguistic similarity, (ii)
Representations of a source language learned by the encoder are dependent on
the target language, and vice-versa, and (iii) Representations of high resource
and/or linguistically similar languages are more robust when fine-tuning on an
arbitrary language pair, which is critical to determining how much
cross-lingual transfer can be expected in a zero or few-shot setting. We
further connect our findings with existing empirical observations in
multilingual NMT and transfer learning.

本研究使用 Singular Value Canonical Correlation Analysis（SVCCA）分析了包含 103 种语言的 NMT 模型，发现不同语言的编码器表示会基于语言相似性聚集，源语言和目标语言的表示相互依赖，并且高资源和 / 或语言相似性更强的语言在任意语言对上进行微调时更为稳健，这些结论对于跨语言转移学习非常重要，并进一步联系到现有的实证观察。

探究规模化多语言 NMT 表示

Investigating Multilingual NMT Representations at Scale

The recently proposed massively multilingual neural machine translation (NMT)
system has been shown to be capable of translating over 100 languages to and
from English within a single model. Its improved translation performance on low
resource languages hints at potential cross-lingual transfer capability for
downstream tasks. In this paper, we evaluate the cross-lingual effectiveness of
representations from the encoder of a massively multilingual NMT model on 5
downstream classification and sequence labeling tasks covering a diverse set of
over 50 languages. We compare against a strong baseline, multilingual BERT
(mBERT), in different cross-lingual transfer learning scenarios and show gains
in zero-shot transfer in 4 out of these 5 tasks.

本文评估了一个大规模多语言神经机器翻译模型编码器在五个跨语言分类和序列标记任务中的跨语言效果，并展示了零 - shot 转移学习中在四个任务中的增益。