A popular approach to creating a zero-shot cross-language retrieval model is
to substitute a monolingual pretrained language model in the retrieval model
with a multilingual pretrained language model such as Multilingual BERT. This
multilingual model is fined-tuned to the retrieval task with monolingual data
such as English MS MARCO using the same training recipe as the monolingual
retrieval model used. However, such transferred models suffer from mismatches
in the languages of the input text during training and inference. In this work,
we propose transferring monolingual retrieval models using adapters, a
parameter-efficient component for a transformer network. By adding adapters
pretrained on language tasks for a specific language with task-specific
adapters, prior work has shown that the adapter-enhanced models perform better
than fine-tuning the entire model when transferring across languages in various
NLP tasks. By constructing dense retrieval models with adapters, we show that
models trained with monolingual data are more effective than fine-tuning the
entire model when transferring to a Cross Language Information Retrieval (CLIR)
setting. However, we found that the prior suggestion of replacing the language
adapters to match the target language at inference time is suboptimal for dense
retrieval models. We provide an in-depth analysis of this discrepancy between
other cross-language NLP tasks and CLIR.

本文研究了利用 adapters 实现单语言检索模型的跨语言迁移，证明了使用 adapters 的密集检索模型在语言迁移方面比整个模型 fine-tuning 更加有效。