The MS MARCO ranking dataset has been widely used for training deep learning
models for IR tasks, achieving considerable effectiveness on diverse zero-shot
scenarios. However, this type of resource is scarce in languages other than
English. In this work, we present mMARCO, a multilingual version of the MS
MARCO passage ranking dataset comprising 13 languages that was created using
machine translation. We evaluated mMARCO by finetuning monolingual and
multilingual reranking models, as well as a multilingual dense retrieval model
on this dataset. We also evaluated models finetuned using the mMARCO dataset in
a zero-shot scenario on Mr. TyDi dataset, demonstrating that multilingual
models finetuned on our translated dataset achieve superior effectiveness to
models finetuned on the original English version alone. Our experiments also
show that a distilled multilingual reranker is competitive with non-distilled
models while having 5.4 times fewer parameters. Lastly, we show a positive
correlation between translation quality and retrieval effectiveness, providing
evidence that improvements in translation methods might lead to improvements in
multilingual information retrieval. The translated datasets and finetuned
models are available at this https URL

该研究介绍了 mMARCO，使用机器翻译创建的包含 13 种语言的 MS MARCO 段落排名数据集的多语言版本，并评估了针对该数据集的多语言重新排名模型和稠密检索模型的效果。研究者们还在零 - shot 场景下使用 mMARCO 数据集微调了模型，在 Mr.TyDi 数据集上展示了多语言模型相对于仅使用英文原版的模型的更高效性。这个研究也表明，翻译质量的提高与检索效果相关，为提高多语版信息检索的效果提供了理论支持。