This paper describes Facebook AI's submission to WMT20 shared news
translation task. We focus on the low resource setting and participate in two
language pairs, Tamil <-> English and Inuktitut <-> English, where there are
limited out-of-domain bitext and monolingual data. We approach the low resource
problem using two main strategies, leveraging all available data and adapting
the system to the target news domain. We explore techniques that leverage
bitext and monolingual data from all languages, such as self-supervised model
pretraining, multilingual models, data augmentation, and reranking. To better
adapt the translation system to the test domain, we explore dataset tagging and
fine-tuning on in-domain data. We observe that different techniques provide
varied improvements based on the available data of the language pair. Based on
the finding, we integrate these techniques into one training pipeline. For
En->Ta, we explore an unconstrained setup with additional Tamil bitext and
monolingual data and show that further improvement can be obtained. On the test
set, our best submitted systems achieve 21.5 and 13.7 BLEU for Ta->En and
En->Ta respectively, and 27.9 and 13.0 for Iu->En and En->Iu respectively.

本篇研究介绍了 Facebook AI 对 WMT20 共享新闻翻译任务的参赛，主要关注于资源有限的环境中，用多种策略和技术如自监督模型预训练、多语言模型、数据增强、标签数据微调等，以对目标新闻领域适应来应对低资源问题。在测试集上，我们的最佳提交系统分别取得 21.5 和 13.7 的 BLEU 得分，适用于 Ta→En 和 En→Ta，以及 27.9 和 13.0 的分数适用于 Iu→En 和 En→Iu。

Facebook AI 的 WMT20 新闻翻译任务提交

Facebook AI's WMT20 News Translation Task Submission

This paper describes Facebook AI's submission to the WAT 2019 Myanmar-English
translation task. Our baseline systems are BPE-based transformer models. We
explore methods to leverage monolingual data to improve generalization,
including self-training, back-translation and their combination. We further
improve results by using noisy channel re-ranking and ensembling. We
demonstrate that these techniques can significantly improve not only a system
trained with additional monolingual data, but even the baseline system trained
exclusively on the provided small parallel dataset. Our system ranks first in
both directions according to human evaluation and BLEU, with a gain of over 8
BLEU points above the second best system.

本文描述了 Facebook AI 在 WAT 2019 缅甸语 - 英语翻译任务中的提交。我们运用基于 BPE 的 transformer 模型，探索了利用单语数据提高泛化能力的方法，包括自训练、回译和它们的组合。我们进一步通过使用噪声通道重排和集成来提高结果。我们证明了这些技术不仅可以显著提高使用额外单语数据训练的系统，而且还可以提高仅使用提供的小型平行数据集训练的基线系统。我们的系统在人类评估和 BLEU 方面均排名第一，在第二名系统上取得了超过 8 BLEU 分的优势。