This paper studies the performances and behaviors of BERT in ranking tasks.
We explore several different ways to leverage the pre-trained BERT and
fine-tune it on two ranking tasks: MS MARCO passage reranking and TREC Web
Track ad hoc document ranking. Experimental results on MS MARCO demonstrate the
strong effectiveness of BERT in question-answering focused passage ranking
tasks, as well as the fact that BERT is a strong interaction-based seq2seq
matching model. Experimental results on TREC show the gaps between the BERT
pre-trained on surrounding contexts and the needs of ad hoc document ranking.
Analyses illustrate how BERT allocates its attentions between query-document
tokens in its Transformer layers, how it prefers semantic matches between
paraphrase tokens, and how that differs with the soft match patterns learned by
a click-trained neural ranker.

本文研究了预先训练的 BERT 在排名任务中的表现和行为，探索了在两种排名任务中优化 BERT 的几种不同方法，并实验证明了 BERT 在问题回答焦点的段落排名任务中的强有效性，以及 BERT 作为强交互式序列到序列匹配模型的事实。

BERT 排名行为解析

Understanding the Behaviors of BERT in Ranking

In order to adopt deep learning for information retrieval, models are needed
that can capture all relevant information required to assess the relevance of a
document to a given user query. While previous works have successfully captured
unigram term matches, how to fully employ position-dependent information such
as proximity and term dependencies has been insufficiently explored. In this
work, we propose a novel neural IR model named PACRR aiming at better modeling
position-dependent interactions between a query and a document. Extensive
experiments on six years' TREC Web Track data confirm that the proposed model
yields better results under multiple benchmarks.

本研究提出了一种名为 PACRR 的新型神经 IR 模型，旨在更好地建模查询和文档之间的位置相关交互，实验结果表明，该模型在多个基准测试下都得到了更好的结果。