Cross-Encoder (CE) and Dual-Encoder (DE) models are two fundamental
approaches for query-document relevance in information retrieval. To predict
relevance, CE models use joint query-document embeddings, while DE models
maintain factorized query and document embeddings; usually, the former has
higher quality while the latter benefits from lower latency. Recently,
late-interaction models have been proposed to realize more favorable
latency-quality tradeoffs, by using a DE structure followed by a lightweight
scorer based on query and document token embeddings. However, these lightweight
scorers are often hand-crafted, and there is no understanding of their
approximation power; further, such scorers require access to individual
document token embeddings, which imposes an increased latency and storage
burden. In this paper, we propose novel learnable late-interaction models
(LITE) that resolve these issues. Theoretically, we prove that LITE is a
universal approximator of continuous scoring functions, even for relatively
small embedding dimension. Empirically, LITE outperforms previous
late-interaction models such as ColBERT on both in-domain and zero-shot
re-ranking tasks. For instance, experiments on MS MARCO passage re-ranking show
that LITE not only yields a model with better generalization, but also lowers
latency and requires 0.25x storage compared to ColBERT.

提出了一种新颖的学习型后交互模型（LITE），通过解决轻量级评分器的手工设计和访问个体文档令牌嵌入的问题来改善查询 - 文档关联性在信息检索中的性能，并在领域内和零样本重排序任务上优于以前的后交互模型。

可学习的后期交互的高效文档排序

Efficient Document Ranking with Learnable Late Interactions

Query-document relevance prediction is a critical problem in Information
Retrieval systems. This problem has increasingly been tackled using
(pretrained) transformer-based models which are finetuned using large
collections of labeled data. However, in specialized domains such as e-commerce
and healthcare, the viability of this approach is limited by the dearth of
large in-domain data. To address this paucity, recent methods leverage these
powerful models to generate high-quality task and domain-specific synthetic
data. Prior work has largely explored synthetic data generation or query
generation (QGen) for Question-Answering (QA) and binary (yes/no) relevance
prediction, where for instance, the QGen models are given a document, and
trained to generate a query relevant to that document. However in many
problems, we have a more fine-grained notion of relevance than a simple yes/no
label. Thus, in this work, we conduct a detailed study into how QGen approaches
can be leveraged for nuanced relevance prediction. We demonstrate that --
contrary to claims from prior works -- current QGen approaches fall short of
the more conventional cross-domain transfer-learning approaches. Via empirical
studies spanning 3 public e-commerce benchmarks, we identify new shortcomings
of existing QGen approaches -- including their inability to distinguish between
different grades of relevance. To address this, we introduce label-conditioned
QGen models which incorporates knowledge about the different relevance. While
our experiments demonstrate that these modifications help improve performance
of QGen techniques, we also find that QGen approaches struggle to capture the
full nuance of the relevance label space and as a result the generated queries
are not faithful to the desired relevance label.

本研究探讨了如何利用 QGen 方法进行细粒度的相关性预测，介绍了 label-conditioned QGen 模型来区分不同等级的相关性，然而 QGen 方法仍然难以捕捉到完整的相关性标签空间，从而生成的查询与所需的相关性标签不符。