Long document re-ranking has been a challenging problem for neural re-rankers
based on deep language models like BERT. Early work breaks the documents into
short passage-like chunks. These chunks are independently mapped to scalar
scores or latent vectors, which are then pooled into a final relevance score.
These encode-and-pool methods however inevitably introduce an information
bottleneck: the low dimension representations. In this paper, we propose
instead to model full query-to-document interaction, leveraging the attention
operation and modular Transformer re-ranker framework. First, document chunks
are encoded independently with an encoder module. An interaction module then
encodes the query and performs joint attention from the query to all document
chunk representations. We demonstrate that the model can use this new degree of
freedom to aggregate important information from the entire document. Our
experiments show that this design produces effective re-ranking on two
classical IR collections Robust04 and ClueWeb09, and a large-scale supervised
collection MS-MARCO document ranking.

本文提出了一种新的长文档重新排序方法，通过利用自注意力机制和模块化 Transformer 框架建立查询到文档的交互模型，避免了编码过程中低维度表示带来的信息瓶颈，实现了从全部文档中提取重要信息的目的，并在 Robust04 和 ClueWeb09 等数据集上实现了有效的重新排序。

使用模块化 Re-ranker 进行长文档重新排名

Long Document Re-ranking with Modular Re-ranker

Ranking responses for a given dialogue context is a popular benchmark in
which the setup is to re-rank the ground-truth response over a limited set of
$n$ responses, where $n$ is typically 10. The predominance of this setup in
conversation response ranking has lead to a great deal of attention to building
neural re-rankers, while the first-stage retrieval step has been overlooked.
Since the correct answer is always available in the candidate list of $n$
responses, this artificial evaluation setup assumes that there is a first-stage
retrieval step which is always able to rank the correct response in its top-$n$
list. In this paper we focus on the more realistic task of full-rank retrieval
of responses, where $n$ can be up to millions of responses. We investigate both
dialogue context and response expansion techniques for sparse retrieval, as
well as zero-shot and fine-tuned dense retrieval approaches. Our findings based
on three different information-seeking dialogue datasets reveal that a learned
response expansion technique is a solid baseline for sparse retrieval. We find
the best performing method overall to be dense retrieval with intermediate
training, i.e. a step after the language model pre-training where sentence
representations are learned, followed by fine-tuning on the target
conversational data. We also investigate the intriguing phenomena that harder
negatives sampling techniques lead to worse results for the fine-tuned dense
retrieval models. The code and datasets are available at
this https URL.

本文针对完整回应的检索任务，提出了对话背景和回应扩充技术的检索方法，探析了稀疏检索和零样本以及微调密集检索方法，并发现通过练习 “学习到的回应扩充技术” 是稀疏检索的一个坚实基础。我们最终发现，整体表现最好的方法是密集的检索中级训练，即是模型先预训练语言模型，再根据目标对话数据进行微调，同时我们研究发现，更难的负采样技术导致微调的稠密点检索模型更糟糕。

对话响应全秩检索的稀疏和密集方法

Sparse and Dense Approaches for the Full-rank Retrieval of Responses for Dialogues

One technique to improve the retrieval effectiveness of a search engine is to
expand documents with terms that are related or representative of the
documents' content.From the perspective of a question answering system, this
might comprise questions the document can potentially answer. Following this
observation, we propose a simple method that predicts which queries will be
issued for a given document and then expands it with those predictions with a
vanilla sequence-to-sequence model, trained using datasets consisting of pairs
of query and relevant documents. By combining our method with a
highly-effective re-ranking component, we achieve the state of the art in two
retrieval tasks. In a latency-critical regime, retrieval results alone (without
re-ranking) approach the effectiveness of more computationally expensive neural
re-rankers but are much faster.

该研究提出了一种将搜索引擎文档扩展为与其内容相关或代表性术语的简单方法，并利用基于 Vanilla 序列到序列模型的数据集对其进行扩展，结合高效的重新排序组件，实现了两种检索任务的最高效结果。