Nearest Neighbor Machine Translation ($k$NN-MT) has achieved great success on
domain adaptation tasks by integrating pre-trained Neural Machine Translation
(NMT) models with domain-specific token-level retrieval. However, the reasons
underlying its success have not been thoroughly investigated. In this paper, we
provide a comprehensive analysis of $k$NN-MT through theoretical and empirical
studies. Initially, we offer a theoretical interpretation of the working
mechanism of $k$NN-MT as an efficient technique to implicitly execute gradient
descent on the output projection layer of NMT, indicating that it is a specific
case of model fine-tuning. Subsequently, we conduct multi-domain experiments
and word-level analysis to examine the differences in performance between
$k$NN-MT and entire-model fine-tuning. Our findings suggest that: (1)
Incorporating $k$NN-MT with adapters yields comparable translation performance
to fine-tuning on in-domain test sets, while achieving better performance on
out-of-domain test sets; (2) Fine-tuning significantly outperforms $k$NN-MT on
the recall of low-frequency domain-specific words, but this gap could be
bridged by optimizing the context representations with additional adapter
layers.

本文分析了 $k$NN-MT 的理论和实证研究，以及针对多域实验和单词级别的分析，发现在特定情况下，结合 $k$NN-MT 和适配器的方法能够实现与在域测试集上微调相当的翻译性能，并在域外测试集上取得更好的性能，同时，优化上下文表示可以弥补低频特定领域词汇召回方面 $k$NN-MT 与微调的差距。

最近邻机器翻译是输出投影层上的元优化器

Nearest Neighbor Machine Translation is Meta-Optimizer on Output  Projection Layer

Though nearest neighbor Machine Translation ($k$NN-MT)
\citep{khandelwal2020nearest} has proved to introduce significant performance
boosts over standard neural MT systems, it is prohibitively slow since it uses
the entire reference corpus as the datastore for the nearest neighbor search.
This means each step for each beam in the beam search has to search over the
entire reference corpus. $k$NN-MT is thus two-orders slower than vanilla MT
models, making it hard to be applied to real-world applications, especially
online services. In this work, we propose Fast $k$NN-MT to address this issue.
Fast $k$NN-MT constructs a significantly smaller datastore for the nearest
neighbor search: for each word in a source sentence, Fast $k$NN-MT first
selects its nearest token-level neighbors, which is limited to tokens that are
the same as the query token. Then at each decoding step, in contrast to using
the entire corpus as the datastore, the search space is limited to target
tokens corresponding to the previously selected reference source tokens. This
strategy avoids search through the whole datastore for nearest neighbors and
drastically improves decoding efficiency. Without loss of performance, Fast
$k$NN-MT is two-orders faster than $k$NN-MT, and is only two times slower than
the standard NMT model. Fast $k$NN-MT enables the practical use of $k$NN-MT
systems in real-world MT applications. The code is available at
https://github.com/ShannonAI/fast-knn-nmt

本文提出了一种快速的 $k$ 最近邻机器翻译模型，通过使用仅限于相同查询令牌的最近令牌级邻居来限制查询空间，从而大大提高了解码效率，并实现了与 $k$NN-MT 相当的性能表现，只比标准 NMT 模型慢两倍。