Neural Networks trained with gradient descent are known to be susceptible to
catastrophic forgetting caused by parameter shift during the training process.
In the context of Neural Machine Translation (NMT) this results in poor
performance on heterogeneous datasets and on sub-tasks like rare phrase
translation. On the other hand, non-parametric approaches are immune to
forgetting, perfectly complementing the generalization ability of NMT. However,
attempts to combine non-parametric or retrieval based approaches with NMT have
only been successful on narrow domains, possibly due to over-reliance on
sentence level retrieval. We propose a novel n-gram level retrieval approach
that relies on local phrase level similarities, allowing us to retrieve
neighbors that are useful for translation even when overall sentence similarity
is low. We complement this with an expressive neural network, allowing our
model to extract information from the noisy retrieved context. We evaluate our
semi-parametric NMT approach on a heterogeneous dataset composed of WMT, IWSLT,
JRC-Acquis and OpenSubtitles, and demonstrate gains on all 4 evaluation sets.
The semi-parametric nature of our approach opens the door for non-parametric
domain adaptation, demonstrating strong inference-time adaptation performance
on new domains without the need for any parameter updates.

本文提出了一种半参数神经机器翻译方法，采用 n-gram 级别的检索方法和表现出色的非参数特性，针对神经网络在训练过程中的参数漂移而提出的而提高网络的鲁棒性的问题，在四种测试数据集上取得了优异的表现，进一步打开了使用非参数方法进行领域自适应的大门。