Long document re-ranking has been a challenging problem for neural re-rankers
based on deep language models like BERT. Early work breaks the documents into
short passage-like chunks. These chunks are independently mapped to scalar
scores or latent vectors, which are then pooled into a final relevance score.
These encode-and-pool methods however inevitably introduce an information
bottleneck: the low dimension representations. In this paper, we propose
instead to model full query-to-document interaction, leveraging the attention
operation and modular Transformer re-ranker framework. First, document chunks
are encoded independently with an encoder module. An interaction module then
encodes the query and performs joint attention from the query to all document
chunk representations. We demonstrate that the model can use this new degree of
freedom to aggregate important information from the entire document. Our
experiments show that this design produces effective re-ranking on two
classical IR collections Robust04 and ClueWeb09, and a large-scale supervised
collection MS-MARCO document ranking.

本文提出了一种新的长文档重新排序方法，通过利用自注意力机制和模块化 Transformer 框架建立查询到文档的交互模型，避免了编码过程中低维度表示带来的信息瓶颈，实现了从全部文档中提取重要信息的目的，并在 Robust04 和 ClueWeb09 等数据集上实现了有效的重新排序。