Cross-encoder (CE) models which compute similarity by jointly encoding a
query-item pair perform better than embedding-based models (dual-encoders) at
estimating query-item relevance. Existing approaches perform k-NN search with
CE by approximating the CE similarity with a vector embedding space fit either
with dual-encoders (DE) or CUR matrix factorization. DE-based
retrieve-and-rerank approaches suffer from poor recall on new domains and the
retrieval with DE is decoupled from the CE. While CUR-based approaches can be
more accurate than the DE-based approach, they require a prohibitively large
number of CE calls to compute item embeddings, thus making it impractical for
deployment at scale. In this paper, we address these shortcomings with our
proposed sparse-matrix factorization based method that efficiently computes
latent query and item embeddings to approximate CE scores and performs k-NN
search with the approximate CE similarity. We compute item embeddings offline
by factorizing a sparse matrix containing query-item CE scores for a set of
train queries. Our method produces a high-quality approximation while requiring
only a fraction of CE calls as compared to CUR-based methods, and allows for
leveraging DE to initialize the embedding space while avoiding compute- and
resource-intensive finetuning of DE via distillation. At test time, the item
embeddings remain fixed and retrieval occurs over rounds, alternating between
a) estimating the test query embedding by minimizing error in approximating CE
scores of items retrieved thus far, and b) using the updated test query
embedding for retrieving more items. Our k-NN search method improves recall by
up to 5% (k=1) and 54% (k=100) over DE-based approaches. Additionally, our
indexing approach achieves a speedup of up to 100x over CUR-based and 5x over
DE distillation methods, while matching or improving k-NN search recall over
baselines.

通过稀疏矩阵因子分解方法，我们的研究提出了一种高效计算潜在查询和项目嵌入以近似 CE 分数的方法，并通过近似 CE 相似性执行 k-NN 搜索。我们的方法在提高检索召回率的同时，实现了 CUR-based 和 DE-based 方法的 100 倍和 5 倍的速度提升。

基于交叉编码器的自适应检索与可扩展索引的 k-NN 搜索

Adaptive Retrieval and Scalable Indexing for k-NN Search with  Cross-Encoders

Cross-encoder models, which jointly encode and score a query-item pair, are
typically prohibitively expensive for k-nearest neighbor search. Consequently,
k-NN search is performed not with a cross-encoder, but with a heuristic
retrieve (e.g., using BM25 or dual-encoder) and re-rank approach. Recent work
proposes ANNCUR (Yadav et al., 2022) which uses CUR matrix factorization to
produce an embedding space for efficient vector-based search that directly
approximates the cross-encoder without the need for dual-encoders. ANNCUR
defines this shared query-item embedding space by scoring the test query
against anchor items which are sampled uniformly at random. While this
minimizes average approximation error over all items, unsuitably high
approximation error on top-k items remains and leads to poor recall of top-k
(and especially top-1) items. Increasing the number of anchor items is a
straightforward way of improving the approximation error and hence k-NN recall
of ANNCUR but at the cost of increased inference latency. In this paper, we
propose a new method for adaptively choosing anchor items that minimizes the
approximation error for the practically important top-k neighbors for a query
with minimal computational overhead. Our proposed method incrementally selects
a suitable set of anchor items for a given test query over several rounds,
using anchors chosen in previous rounds to inform selection of more anchor
items. Empirically, our method consistently improves k-NN recall as compared to
both ANNCUR and the widely-used dual-encoder-based retrieve-and-rerank
approach.

本研究提出了一种针对实际中 k-NN 查询困难的问题，采用近似矩阵分解技术的方法进行高效向量检索，该方法可以自适应地选择锚定项，以最小化检索近似误差。研究实验证明，该方法相对于使用 BM25 或双编码器的传统检索方法和 ANNCUR 方法，能够提高 k-NN 检索率。

基于 CUR 的 k-NN 搜索与交叉编码器的自适应锚定项选择

Adaptive Selection of Anchor Items for CUR-based k-NN search with  Cross-Encoders

Pairing a lexical retriever with a neural re-ranking model has set
state-of-the-art performance on large-scale information retrieval datasets.
This pipeline covers scenarios like question answering or navigational queries,
however, for information-seeking scenarios, users often provide information on
whether a document is relevant to their query in form of clicks or explicit
feedback. Therefore, in this work, we explore how relevance feedback can be
directly integrated into neural re-ranking models by adopting few-shot and
parameter-efficient learning techniques. Specifically, we introduce a kNN
approach that re-ranks documents based on their similarity with the query and
the documents the user considers relevant. Further, we explore Cross-Encoder
models that we pre-train using meta-learning and subsequently fine-tune for
each query, training only on the feedback documents. To evaluate our different
integration strategies, we transform four existing information retrieval
datasets into the relevance feedback scenario. Extensive experiments
demonstrate that integrating relevance feedback directly in neural re-ranking
models improves their performance, and fusing lexical ranking with our best
performing neural re-ranker outperforms all other methods by 5.2 nDCG@20.

本研究探讨如何将相关性反馈直接集成到神经重排模型中，使用 kNN 方法和交叉编码器模型，通过在现有信息检索数据集上的实验，证明了这种方法能够大幅提高神经重排模型的性能。