Translating text that diverges from the training domain is a key challenge for neural machine translation (NMT). Domain robustness - the generalization of models to unseen test domains - is low compared to statistical machine translation. In this paper, we investigate the performance of NMT on out-of-domain test sets, and ways to improve it. We observe that hallucination (translations that are fluent but unrelated to the source) is common in out-of-domain settings, and we empirically compare methods that improve adequacy (reconstruction), out-of-domain translation (subword regularization), or robustness against adversarial examples (defensive distillation), as well as noisy channel models. In experiments on German to English OPUS data, and German to Romansh, a low-resource scenario, we find that several methods improve domain robustness, reconstruction standing out as a method that not only improves automatic scores, but also shows improvements in a manual assessments of adequacy, albeit at some loss in fluency. However, out-of-domain performance is still relatively low and domain robustness remains an open problem.

本文研究了机器翻译模型在未知领域中的性能表现，发现在未知领域中，统计机器翻译（SMT）和神经机器翻译（NMT）系统存在不同的问题：SMT系统通常足够但不流利，而NMT系统流利但不足够。本文发现，NMT系统出现的“幻觉”现象是造成低域稳健性的主要原因，研究并比较了有关提高NMT系统稳健性的方法，实验结果表明，这些方法能够稍微提高NMT系统翻译的充分性，但对流畅性的提高较小。

神经机器翻译的领域鲁棒性