Building dense retrievers requires a series of standard procedures, including
training and validating neural models and creating indexes for efficient
search. However, these procedures are often misaligned in that training
objectives do not exactly reflect the retrieval scenario at inference time. In
this paper, we explore how the gap between training and inference in dense
retrieval can be reduced, focusing on dense phrase retrieval (Lee et al., 2021)
where billions of representations are indexed at inference. Since validating
every dense retriever with a large-scale index is practically infeasible, we
propose an efficient way of validating dense retrievers using a small subset of
the entire corpus. This allows us to validate various training strategies
including unifying contrastive loss terms and using hard negatives for phrase
retrieval, which largely reduces the training-inference discrepancy. As a
result, we improve top-1 phrase retrieval accuracy by 2~3 points and top-20
passage retrieval accuracy by 2~4 points for open-domain question answering.
Our work urges modeling dense retrievers with careful consideration of training
and inference via efficient validation while advancing phrase retrieval as a
general solution for dense retrieval.

本研究探讨了如何通过有效验证缩小训练与检索之间的差距并提高稠密检索的性能，在密集短语检索中取得了 2~3 个关键短语检索准确度和 2~4 个段落检索准确度的提升。

通过桥接训练 - 推理差距实现密集短语检索

Bridging the Training-Inference Gap for Dense Phrase Retrieval

Dense retrieval methods have shown great promise over sparse retrieval
methods in a range of NLP problems. Among them, dense phrase retrieval-the most
fine-grained retrieval unit-is appealing because phrases can be directly used
as the output for question answering and slot filling tasks. In this work, we
follow the intuition that retrieving phrases naturally entails retrieving
larger text blocks and study whether phrase retrieval can serve as the basis
for coarse-level retrieval including passages and documents. We first observe
that a dense phrase-retrieval system, without any retraining, already achieves
better passage retrieval accuracy (+3-5% in top-5 accuracy) compared to passage
retrievers, which also helps achieve superior end-to-end QA performance with
fewer passages. Then, we provide an interpretation for why phrase-level
supervision helps learn better fine-grained entailment compared to
passage-level supervision, and also show that phrase retrieval can be improved
to achieve competitive performance in document-retrieval tasks such as entity
linking and knowledge-grounded dialogue. Finally, we demonstrate how phrase
filtering and vector quantization can reduce the size of our index by 4-10x,
making dense phrase retrieval a practical and versatile solution in
multi-granularity retrieval.

通过研究密集短语检索系统，探讨短语检索是否可以作为粗粒度检索（如段落和文档）的基础，并提出了短语过滤和矢量量化等方法，可以将检索索引的大小减小至原来的 4-10 倍，从而使得密集短语检索成为多粒度检索的实用解决方案。