In open-domain question answering, dense passage retrieval has become a new
paradigm to retrieve relevant passages for finding answers. Typically, the
dual-encoder architecture is adopted to learn dense representations of
questions and passages for semantic matching. However, it is difficult to
effectively train a dual-encoder due to the challenges including the
discrepancy between training and inference, the existence of unlabeled
positives and limited training data. To address these challenges, we propose an
optimized training approach, called RocketQA, to improving dense passage
retrieval. We make three major technical contributions in RocketQA, namely
cross-batch negatives, denoised hard negatives and data augmentation. The
experiment results show that RocketQA significantly outperforms previous
state-of-the-art models on both MSMARCO and Natural Questions. We also conduct
extensive experiments to examine the effectiveness of the three strategies in
RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be
improved based on our RocketQA retriever.

本文提出了一种优化的训练方法 RocketQA 来改进密集问答检索，通过跨批次负样例、去噪硬负样例和数据增强等三种策略的实验，证明该方法在 MSMARCO 和 Natural Questions 上都显著优于之前的最先进模型，并提高了 end-to-end QA 的性能。