We investigate techniques for supervised domain adaptation for neural machine translation where an existing model trained on a large out-of-domain dataset is adapted to a small in-domain dataset. In this scenario, overfitting is a major challenge. We investigate a number of techniques to reduce overfitting and improve transfer learning, including regularization techniques such as dropout and L2-regularization towards an out-of-domain prior. In addition, we introduce tuneout, a novel regularization technique inspired by dropout. We apply these techniques, alone and in combination, to neural machine translation, obtaining improvements on IWSLT datasets for English->German and English->Russian. We also investigate the amounts of in-domain training data needed for domain adaptation in NMT, and find a logarithmic relationship between the amount of training data and gain in BLEU score.

本研究探讨了神经机器翻译的监督域自适应技术，针对现有模型在大型非领域数据集训练后，如何适应小型领域数据集所出现的过拟合问题。在整个过程中，正则化技术，如dropout和L2正则化与非领域先验之间的关系得到了深入研究。此外，文章提出了一种新型的正则化技术——tuneout，即启发式dropout。我们将这些技术单独或结合应用于神经机器翻译，以英语->德语、英语->俄语的IWSLT数据集为例取得了改进。此外，我们还研究了NMT领域中所需的领域内训练数据量，并发现培训数据与BLEU分数之间存在对数关系。

神经机器翻译微调的正则化技术