Current dense text retrieval models face two typical challenges. First, they
adopt a siamese dual-encoder architecture to encode queries and documents
independently for fast indexing and searching, while neglecting the
finer-grained term-wise interactions. This results in a sub-optimal recall
performance. Second, their model training highly relies on a negative sampling
technique to build up the negative documents in their contrastive losses. To
address these challenges, we present Adversarial Retriever-Ranker (AR2), which
consists of a dual-encoder retriever plus a cross-encoder ranker. The two
models are jointly optimized according to a minimax adversarial objective: the
retriever learns to retrieve negative documents to cheat the ranker, while the
ranker learns to rank a collection of candidates including both the
ground-truth and the retrieved ones, as well as providing progressive direct
feedback to the dual-encoder retriever. Through this adversarial game, the
retriever gradually produces harder negative documents to train a better
ranker, whereas the cross-encoder ranker provides progressive feedback to
improve retriever. We evaluate AR2 on three benchmarks. Experimental results
show that AR2 consistently and significantly outperforms existing dense
retriever methods and achieves new state-of-the-art results on all of them.
This includes the improvements on Natural Questions R@5 to 77.9%(+2.1%),
TriviaQA R@5 to 78.2%(+1.4), and MS-MARCO MRR@10 to 39.5%(+1.3%). Code and
models are available at this https URL

提出 Adversarial Retriever-Ranker（AR2），其中包括双编码器 Retrieval Model 和交叉编码器 Ranking Model，以解决当前的稠密文本检索模型面临的两个主要挑战，既快速索引和搜索，也考虑更细粒度的词级相互作用，同时为了建立负面文件，其模型训练高度依赖于负采样技术。在三个基准测试中，实验结果显示 AR2 在所有测试中 consistently and significantly outperforms 现有的关键词检索方法并 achievs 新的 state-of-the-art 结果。