Ranking has always been one of the top concerns in information retrieval
researches. For decades, the lexical matching signal has dominated the ad-hoc
retrieval process, but solely using this signal in retrieval may cause the
vocabulary mismatch problem. In recent years, with the development of
representation learning techniques, many researchers turn to Dense Retrieval
(DR) models for better ranking performance. Although several existing DR models
have already obtained promising results, their performance improvement heavily
relies on the sampling of training examples. Many effective sampling strategies
are not efficient enough for practical usage, and for most of them, there still
lacks theoretical analysis in how and why performance improvement happens. To
shed light on these research questions, we theoretically investigate different
training strategies for DR models and try to explain why hard negative sampling
performs better than random sampling. Through the analysis, we also find that
there are many potential risks in static hard negative sampling, which is
employed by many existing training methods. Therefore, we propose two training
strategies named a Stable Training Algorithm for dense Retrieval (STAR) and a
query-side training Algorithm for Directly Optimizing Ranking pErformance
(ADORE), respectively. STAR improves the stability of DR training process by
introducing random negatives. ADORE replaces the widely-adopted static hard
negative sampling method with a dynamic one to directly optimize the ranking
performance. Experimental results on two publicly available retrieval benchmark
datasets show that either strategy gains significant improvements over existing
competitive baselines and a combination of them leads to the best performance.

本研究研究了 Dense Retrieval (DR) 模型训练中的不同采样策略，并解释了为何硬负采样优于随机采样。研究发现目前许多基于硬负采样的训练方法存在许多潜在风险，因此本文提出了 Stable Training Algorithm (STAR) 和 Query-side Training Algorithm for Directly Optimizing Ranking Performance (ADORE) 两种训练策略，并在两个公共检索基准数据集上进行了实验，结果表明这两种策略均取得了显著的改进，并且结合这两种策略可以达到最佳性能。