Pseudo-relevance feedback mechanisms, from Rocchio to the relevance models,
have shown the usefulness of expanding and reweighting the users' initial
queries using information occurring in an initial set of retrieved documents,
known as the pseudo-relevant set. Recently, dense retrieval -- through the use
of neural contextual language models such as BERT for analysing the documents'
and queries' contents and computing their relevance scores -- has shown a
promising performance on several information retrieval tasks still relying on
the traditional inverted index for identifying documents relevant to a query.
Two different dense retrieval families have emerged: the use of single embedded
representations for each passage and query (e.g. using BERT's [CLS] token), or
via multiple representations (e.g. using an embedding for each token of the
query and document). In this work, we conduct the first study into the
potential for multiple representation dense retrieval to be enhanced using
pseudo-relevance feedback. In particular, based on the pseudo-relevant set of
documents identified using a first-pass dense retrieval, we extract
representative feedback embeddings (using KMeans clustering) -- while ensuring
that these embeddings discriminate among passages (based on IDF) -- which are
then added to the query representation. These additional feedback embeddings
are shown to both enhance the effectiveness of a reranking as well as an
additional dense retrieval operation. Indeed, experiments on the MSMARCO
passage ranking dataset show that MAP can be improved by upto 26% on the TREC
2019 query set and 10% on the TREC 2020 query set by the application of our
proposed ColBERT-PRF method on a ColBERT dense retrieval approach.

本研究探讨了基于伪相关反馈的多重嵌入密集检索方法的潜力。通过从第一遍密集检索中确定的伪相关文件集中提取代表性反馈嵌入，然后将其添加到查询表示中，证明了这些附加反馈嵌入不仅能够提高重新排序的有效性，还能提高附加密集检索操作的效率。