Benefiting from transformer-based pre-trained language models, neural ranking
models have made significant progress. More recently, the advent of
multilingual pre-trained language models provides great support for designing
neural cross-lingual retrieval models. However, due to unbalanced pre-training
data in different languages, multilingual language models have already shown a
performance gap between high and low-resource languages in many downstream
tasks. And cross-lingual retrieval models built on such pre-trained models can
inherit language bias, leading to suboptimal result for low-resource languages.
Moreover, unlike the English-to-English retrieval task, where large-scale
training collections for document ranking such as MS MARCO are available, the
lack of cross-lingual retrieval data for low-resource language makes it more
challenging for training cross-lingual retrieval models. In this work, we
propose OPTICAL: Optimal Transport distillation for low-resource Cross-lingual
information retrieval. To transfer a model from high to low resource languages,
OPTICAL forms the cross-lingual token alignment task as an optimal transport
problem to learn from a well-trained monolingual retrieval model. By separating
the cross-lingual knowledge from knowledge of query document matching, OPTICAL
only needs bitext data for distillation training, which is more feasible for
low-resource languages. Experimental results show that, with minimal training
data, OPTICAL significantly outperforms strong baselines on low-resource
languages, including neural machine translation.

本文提出了 OPTICAL：优化传输蒸馏用于低资源跨语种信息检索，该方法利用优化传输问题将知识从双语数据中分离出来，只需要少量数据就可以在低资源语种上显著提高效果。