Transformers (Vaswani et al., 2017) have brought a remarkable improvement in
the performance of neural machine translation (NMT) systems but they could be
surprisingly vulnerable to noise. In this work, we try to investigate how noise
breaks Transformers and if there exist solutions to deal with such issues.
There is a large body of work in the NMT literature on analyzing the behavior
of conventional models for the problem of noise but Transformers are relatively
understudied in this context. Motivated by this, we introduce a novel
data-driven technique called Target Augmented Fine-tuning (TAFT) to incorporate
noise during training. This idea is comparable to the well-known fine-tuning
strategy. Moreover, we propose two other novel extensions to the original
Transformer: Controlled Denoising (CD) and Dual-Channel Decoding (DCD), that
modify the neural architecture as well as the training process to handle noise.
One important characteristic of our techniques is that they only impact the
training phase and do not impose any overhead at inference time. We evaluated
our techniques to translate the English--German pair in both directions and
observed that our models have a higher tolerance to noise. More specifically,
they perform with no deterioration where up to 10% of entire test words are
infected by noise.

本文介绍了一种名为 TAFT 的数据驱动技术，它利用 fine-tuning 策略加入噪声训练 Transformer 模型，并提出了两种新型技术 CD 和 DCD 以帮助模型更好地处理噪声，最终在英德翻译语料中实现更高的鲁棒性。

重新审视鲁棒神经机器翻译：基于 Transformer 的案例研究

Revisiting Robust Neural Machine Translation: A Transformer Case Study

This paper describes the Microsoft submission to the WMT2018 news translation
shared task. We participated in one language direction -- English-German. Our
system follows current best-practice and combines state-of-the-art models with
new data filtering (dual conditional cross-entropy filtering) and sentence
weighting methods. We trained fairly standard Transformer-big models with an
updated version of Edinburgh's training scheme for WMT2017 and experimented
with different filtering schemes for Paracrawl. According to automatic metrics
(BLEU) we reached the highest score for this subtask with a nearly 2 BLEU point
margin over the next strongest system. Based on human evaluation we ranked
first among constrained systems. We believe this is mostly caused by our data
filtering/weighting regime.

本文介绍了微软参加 WMT2018 新闻翻译共享任务的提交，对于英语到德语的一种语言方向中，基于最佳实践，采用了最先进的模型和新数据过滤和句子加权方法来训练在 Paracrawl 上实验结果表明，根据自动指标 (BLEU)，我们在此子任务中得分最高，比接下来最强大的系统高近 2 BLEU 分数，根据人工评估，我们在受限系统中排名第一，我们认为这主要是由于我们的数据过滤 / 加权组合的实施。