Recent work in cross-language information retrieval (CLIR), where queries and
documents are in different languages, has shown the benefit of the
Translate-Distill framework that trains a cross-language neural dual-encoder
model using translation and distillation. However, Translate-Distill only
supports a single document language. Multilingual information retrieval (MLIR),
which ranks a multilingual document collection, is harder to train than CLIR
because the model must assign comparable relevance scores to documents in
different languages. This work extends Translate-Distill and propose
Multilingual Translate-Distill (MTD) for MLIR. We show that ColBERT-X models
trained with MTD outperform their counterparts trained ith Multilingual
Translate-Train, which is the previous state-of-the-art training approach, by
5% to 25% in nDCG@20 and 15% to 45% in MAP. We also show that the model is
robust to the way languages are mixed in training batches. Our implementation
is available on GitHub.

使用 Translate-Distill 框架进行跨语言信息检索 (CLIR) 已经显示出其通过翻译和塑造训练跨语言神经双重编码器模型的好处。然而，Translate-Distill 仅支持单一语言。本研究扩展 Translate-Distill 并提出了用于多语言信息检索 (MLIR) 的多语言 Translate-Distill (MTD) 方法。我们通过 nDCG@20 指标和 MAP 指标在性能上展示了使用 MTD 培训的 ColBERT-X 模型优于以前的最先进训练方法 Multilingual Translate-Train，其在 nDCG@20 上提高了 5% 到 25%，在 MAP 上提高了 15% 到 45%。我们还展示了该模型对于训练批次中语言混合方式的鲁棒性。我们的实现可在 GitHub 上找到。

多语言信息检索的蒸馏

Distillation for Multilingual Information Retrieval

Prior work on English monolingual retrieval has shown that a cross-encoder
trained using a large number of relevance judgments for query-document pairs
can be used as a teacher to train more efficient, but similarly effective,
dual-encoder student models. Applying a similar knowledge distillation approach
to training an efficient dual-encoder model for Cross-Language Information
Retrieval (CLIR), where queries and documents are in different languages, is
challenging due to the lack of a sufficiently large training collection when
the query and document languages differ. The state of the art for CLIR thus
relies on translating queries, documents, or both from the large English MS
MARCO training set, an approach called Translate-Train. This paper proposes an
alternative, Translate-Distill, in which knowledge distillation from either a
monolingual cross-encoder or a CLIR cross-encoder is used to train a
dual-encoder CLIR student model. This richer design space enables the teacher
model to perform inference in an optimized setting, while training the student
model directly for CLIR. Trained models and artifacts are publicly available on
Huggingface.

该研究提出了一种名为 Translate-Distill 的方法，它利用交叉编码器或 CLIR 交叉编码器的知识蒸馏来训练双编码器 CLIR 学生模型。