The Transformer-Kernel (TK) model has demonstrated strong reranking
performance on the TREC Deep Learning benchmark---and can be considered to be
an efficient (but slightly less effective) alternative to BERT-based ranking
models. In this work, we extend the TK architecture to the full retrieval
setting by incorporating the query term independence assumption. Furthermore,
to reduce the memory complexity of the Transformer layers with respect to the
input sequence length, we propose a new Conformer layer. We show that the
Conformer's GPU memory requirement scales linearly with input sequence length,
making it a more viable option when ranking long documents. Finally, we
demonstrate that incorporating explicit term matching signal into the model can
be particularly useful in the full retrieval setting. We present preliminary
results from our work in this paper.

本文所研究的 Transformer-Kernel 模型已经在 TREC Deep Learning benchmark 中表现出强大的重排名表现，它是 BERT 排名模型的有效但略微不够有效的替代品。本文将 TK 体系结构扩展到全检索设置并考虑查询术语独立假设，提出将 Conformer 层结合在一起，同时还演示了将显式术语匹配信号纳入模型可能在全检索设置中特别有用的初步结果。

Conformer-Kernel 带有查询项独立性的文档检索

Conformer-Kernel with Query Term Independence for Document Retrieval

Search engines operate under a strict time constraint as a fast response is
paramount to user satisfaction. Thus, neural re-ranking models have a limited
time-budget to re-rank documents. Given the same amount of time, a faster
re-ranking model can incorporate more documents than a less efficient one,
leading to a higher effectiveness. To utilize this property, we propose TK
(Transformer-Kernel): a neural re-ranking model for ad-hoc search using an
efficient contextualization mechanism. TK employs a very small number of
Transformer layers (up to three) to contextualize query and document word
embeddings. To score individual term interactions, we use a document-length
enhanced kernel-pooling, which enables users to gain insight into the model. TK
offers an optimal ratio between effectiveness and efficiency: under realistic
time constraints (max. 200 ms per query) TK achieves the highest effectiveness
in comparison to BERT and other re-ranking models. We demonstrate this on three
large-scale ranking collections: MSMARCO-Passage, MSMARCO-Document, and TREC
CAR. In addition, to gain insight into TK, we perform a clustered query
analysis of TK's results, highlighting its strengths and weaknesses on queries
with different types of information need and we show how to interpret the cause
of ranking differences of two documents by comparing their internal scores.

提出了一种名为 TK（Transformer-Kernel）的神经再排序模型，利用一种高效的上下文化机制来进行广告搜索。在最大每次查询 200 毫秒的实际时间限制下，TK 比 BERT 和其他排序模型实现了最高的有效性，通过对 TK 的结果进行聚类查询分析，突出了它在具有不同信息需求类型的查询中的优势和劣势，并展示了通过比较两个文档的内部分数来解释排名差异的原因。