Neural retrievers based on dense representations combined with Approximate
Nearest Neighbors search have recently received a lot of attention, owing their
success to distillation and/or better sampling of examples for training --
while still relying on the same backbone architecture. In the meantime, sparse
representation learning fueled by traditional inverted indexing techniques has
seen a growing interest, inheriting from desirable IR priors such as explicit
lexical matching. While some architectural variants have been proposed, a
lesser effort has been put in the training of such models. In this work, we
build on SPLADE -- a sparse expansion-based retriever -- and show to which
extent it is able to benefit from the same training improvements as dense
models, by studying the effect of distillation, hard-negative mining as well as
the Pre-trained Language Model initialization. We furthermore study the link
between effectiveness and efficiency, on in-domain and zero-shot settings,
leading to state-of-the-art results in both scenarios for sufficiently
expressive models.

本文研究基于稀疏表达的 SPLADE 检索器的训练提升方法，结合蒸馏、硬负例挖掘以及预训练语言模型初始化，证明其在效率和效果方面同样具有优势，能够在领域内和零样本情况下实现最先进的结果。