NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS)
models flounder when sufficient amounts of parallel data is not available for
fine-tuning. This specifically holds for languages missing/under-represented in
these models. The problem gets aggravated when the data comes from different
domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of
PMSS models is extremely beneficial for domain-specific NMT, especially when
target domain data is limited/unavailable and the considered languages are
missing or under-represented in the PMSS model. We quantify the domain-specific
results variations using a domain-divergence test, and show that ITFT can
mitigate the impact of domain divergence to some extent.

本文研究表明，在缺乏足够的平行数据进行微调的情况下，基于预训练多语言序列 - 序列（PMSS）模型训练的 NMT 系统会受到影响，尤其是这些模型中缺少或代表不足的语言。当数据来自不同的领域时，问题会加剧。本文表明，对 PMSS 模型进行中间任务微调（ITFT）对于特定领域的 NMT 非常有益，特别是当目标领域数据有限 / 不可用并且所考虑的语言在 PMSS 模型中缺失或代表不足时。我们使用领域分歧测试量化领域特定的结果变化，并显示 ITFT 可以在一定程度上缓解领域分歧的影响。