This paper provides an overview of NVIDIA NeMo's neural machine translation
systems for the constrained data track of the WMT21 News and Biomedical Shared
Translation Tasks. Our news task submissions for English-German (En-De) and
English-Russian (En-Ru) are built on top of a baseline transformer-based
sequence-to-sequence model. Specifically, we use a combination of 1) checkpoint
averaging 2) model scaling 3) data augmentation with backtranslation and
knowledge distillation from right-to-left factorized models 4) finetuning on
test sets from previous years 5) model ensembling 6) shallow fusion decoding
with transformer language models and 7) noisy channel re-ranking. Additionally,
our biomedical task submission for English-Russian uses a biomedically biased
vocabulary and is trained from scratch on news task data, medically relevant
text curated from the news task dataset, and biomedical data provided by the
shared task. Our news system achieves a sacreBLEU score of 39.5 on the WMT'20
En-De test set outperforming the best submission from last year's task of 38.8.
Our biomedical task Ru-En and En-Ru systems reach BLEU scores of 43.8 and 40.3
respectively on the WMT'20 Biomedical Task Test set, outperforming the previous
year's best submissions.

本文介绍了 NVIDIA NeMo 神经机器翻译系统在 WMT21 新闻和生物医学共享翻译任务的受限数据跟踪中的运用，其中新闻任务的提交在 Transform 器基础的序列到序列模型上，采用了多项技术手段并且获得了比去年任务最佳提交更好的表现，生物医学任务的提交则采取了医学相关文本来源和生物医学数据的训练集，而两个系统都在测试集上超越了去年任务的最佳提交。

NVIDIA NeMo WMT21 英德及英俄新闻和生物医学任务神经机器翻译系统

NVIDIA NeMo Neural Machine Translation Systems for English-German and  English-Russian News and Biomedical Tasks at WMT21

This report describes Microsoft's machine translation systems for the WMT21
shared task on large-scale multilingual machine translation. We participated in
all three evaluation tracks including Large Track and two Small Tracks where
the former one is unconstrained and the latter two are fully constrained. Our
model submissions to the shared task were initialized with
DeltaLM\footnote{https://aka.ms/deltalm}, a generic pre-trained
multilingual encoder-decoder model, and fine-tuned correspondingly with the
vast collected parallel data and allowed data sources according to track
settings, together with applying progressive learning and iterative
back-translation approaches to further improve the performance. Our final
submissions ranked first on three tracks in terms of the automatic evaluation
metric.

本文介绍 Microsoft 在 WMT21 大规模多语言机器翻译竞赛中使用的机器翻译系统，该系统使用基于 DeltaLM 的泛用预训练多语言编码器 - 解码器模型进行初始化，应用迭代反向翻译方法和渐进式学习来优化性能，在三个评估轨迹中都获得了第一名。