Neural Information Retrieval models hold the promise to replace lexical
matching models, e.g. BM25, in modern search engines. While their capabilities
have fully shone on in-domain datasets like MS MARCO, they have recently been
challenged on out-of-domain zero-shot settings (BEIR benchmark), questioning
their actual generalization capabilities compared to bag-of-words approaches.
Particularly, we wonder if these shortcomings could (partly) be the consequence
of the inability of neural IR models to perform lexical matching off-the-shelf.
In this work, we propose a measure of discrepancy between the lexical matching
performed by any (neural) model and an 'ideal' one. Based on this, we study the
behavior of different state-of-the-art neural IR models, focusing on whether
they are able to perform lexical matching when it's actually useful, i.e. for
important terms. Overall, we show that neural IR models fail to properly
generalize term importance on out-of-domain collections or terms almost unseen
during training

本文研究了神经信息检索模型在词汇匹配方面的能力，提出了与 “理想” 匹配性的不一致度量，并研究了不同的最先进的神经信息检索模型的行为。结果表明，这些模型在对于重要术语时无法实现词汇匹配，从而在跨领域检索上表现不佳。