k-nearest-neighbor machine translation has demonstrated remarkable
improvements in machine translation quality by creating a datastore of cached
examples. However, these improvements have been limited to high-resource
language pairs, with large datastores, and remain a challenge for low-resource
languages. In this paper, we address this issue by combining representations
from multiple languages into a single datastore. Our results consistently
demonstrate substantial improvements not only in low-resource translation
quality (up to +3.6 BLEU), but also for high-resource translation quality (up
to +0.5 BLEU). Our experiments show that it is possible to create multilingual
datastores that are a quarter of the size, achieving a 5.3x speed improvement,
by using linguistic similarities for datastore creation.

通过使用多种语言的表示方法结合成一个数据存储，我们可以在低资源翻译质量上获得显著改进（高达 + 3.6 BLEU），同时对高资源翻译质量也有所提高（高达 + 0.5 BLEU）。我们的实验证实了通过使用语言相似性进行数据存储创建，可以实现四分之一大小的多语言数据存储，从而提高了翻译速度（提升了 5.3 倍）.

多语种 K 最近邻机器翻译

Multilingual k-Nearest-Neighbor Machine Translation

k-nearest-neighbor machine translation (kNN-MT) boosts the translation
quality of a pre-trained neural machine translation (NMT) model by utilizing
translation examples during decoding. Translation examples are stored in a
vector database, called a datastore, which contains one entry for each target
token from the parallel data it is made from. Due to its size, it is
computationally expensive both to construct and to retrieve examples from the
datastore. In this paper, we present an efficient and extensible kNN-MT
framework, knn-seq, for researchers and developers that is carefully designed
to run efficiently, even with a billion-scale large datastore. knn-seq is
developed as a plug-in on fairseq and easy to switch models and kNN indexes.
Experimental results show that our implemented kNN-MT achieves a comparable
gain to the original kNN-MT, and the billion-scale datastore construction took
2.21 hours in the WMT'19 German-to-English translation task. We publish our
knn-seq as an MIT-licensed open-source project and the code is available on
this https URL . The demo video is available on
this https URL .

k-nearest-neighbor machine translation improves neural machine translation by utilizing a vector database of translation examples, and the paper presents an efficient framework called knn-seq to handle large datasets.

knn-seq: 高效，可扩展的 kNN-MT 框架

knn-seq: Efficient, Extensible kNN-MT Framework

Fine-tuning a language model on a new domain is standard practice for domain
adaptation. However, it can be infeasible when it comes to modern large-scale
language models such as GPT-3, which can only be accessed through APIs, making
it difficult to access the internal parameters of the model. In this paper, we
propose $k$NN-Adapter, a method to effectively adapt these black-box large
language models (LLMs) to a new domain. The $k$NN-Adapter builds on top of the
retrieval-augmented language model, and adaptively learns to interpolate the
output of the language model with retrieval results from a datastore consisting
of the target domain data. Our experiments on four different domains
demonstrate that $k$NN-Adapter significantly improves perplexity, and works
particularly well in settings with limited access to LLMs. Additionally, we
show that $k$NN-Adapter is more effective than fine-tuning when the amount of
training data is limited. We also release a dataset to encourage further study.

本文介绍了 $k$NN-Adapter 方法，该方法是基于检索增强语言模型，通过将语言模型的输出和目标域数据存储中的检索结果进行自适应加权插值，有效地适应黑盒大型语言模型到新的领域。实验结果表明，$k$NN-Adapter 方法显著提高了语言模型的困惑度，在训练数据有限的情况下比微调更有效。

kNN-Adapter: 黑盒语言模型的高效领域自适应

$k$NN-Adapter: Efficient Domain Adaptation for Black-Box Language Models

Recent advances in image captioning have focused on scaling the data and
model size, substantially increasing the cost of pre-training and finetuning.
As an alternative to large models, we present SmallCap, which generates a
caption conditioned on an input image and related captions retrieved from a
datastore. Our model is lightweight and fast to train as the only learned
parameters are in newly introduced cross-attention layers between a pre-trained
CLIP encoder and GPT-2 decoder. SmallCap can transfer to new domains without
additional finetuning and exploit large-scale data in a training-free fashion
because the contents of the datastore can be readily replaced. Our experiments
show that SmallCap, trained only on COCO, has competitive performance on this
benchmark, and also transfers to other domains without retraining, solely
through retrieval from target-domain data. Further improvement is achieved
through the training-free exploitation of diverse human-labeled and web data,
which proves effective for other domains, including the nocaps image captioning
benchmark, designed to test generalization to unseen visual concepts.

SmallCap 是一个轻量且快速的图像字幕生成模型，它通过用与目标图像相关的字幕在数据存储中进行训练，而无需对模型进行额外的微调以实现跨域迁移学习，同时避免了扩大模型规模的开销。