To alleviate the data scarcity problem in training question answering
systems, recent works propose additional intermediate pre-training for dense
passage retrieval (DPR). However, there still remains a large discrepancy
between the provided upstream signals and the downstream question-passage
relevance, which leads to less improvement. To bridge this gap, we propose the
HyperLink-induced Pre-training (HLP), a method to pre-train the dense retriever
with the text relevance induced by hyperlink-based topology within Web
documents. We demonstrate that the hyperlink-based structures of dual-link and
co-mention can provide effective relevance signals for large-scale pre-training
that better facilitate downstream passage retrieval. We investigate the
effectiveness of our approach across a wide range of open-domain QA datasets
under zero-shot, few-shot, multi-hop, and out-of-domain scenarios. The
experiments show our HLP outperforms the BM25 by up to 7 points as well as
other pre-training methods by more than 10 points in terms of top-20 retrieval
accuracy under the zero-shot scenario. Furthermore, HLP significantly
outperforms other pre-training methods under the other scenarios.

提出 HyperLink-induced Pre-training (HLP) 方法来解决文档中基于超链接拓扑结构的相关性所面临的问题，为大规模预训练提供更加有效的相关性信号，从而使得在问答过程中的段落检索能够被更好地实现。在各种场景下，HLP 方法都优于其他预训练方法。

超链接诱导的预训练：面向开放领域问答的段落检索

Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering

We present Mr. TyDi, a multi-lingual benchmark dataset for mono-lingual
retrieval in eleven typologically diverse languages, designed to evaluate
ranking with learned dense representations. The goal of this resource is to
spur research in dense retrieval techniques in non-English languages, motivated
by recent observations that existing techniques for representation learning
perform poorly when applied to out-of-distribution data. As a starting point,
we provide zero-shot baselines for this new dataset based on a multi-lingual
adaptation of DPR that we call "mDPR". Experiments show that although the
effectiveness of mDPR is much lower than BM25, dense representations
nevertheless appear to provide valuable relevance signals, improving BM25
results in sparse-dense hybrids. In addition to analyses of our results, we
also discuss future challenges and present a research agenda in multi-lingual
dense retrieval. Mr. TyDi can be downloaded at
this https URL

介绍了 Mr.TyDi - 一个包括 11 种不同语言、多语言检索基准数据集，旨在评估利用学习的密集表示进行排名的效果。通过提供多语言 DPR 等基线模型，实验结果表明，虽然效果比 BM25 模型要差，但密集表示仍然可以提供有价值的相关信号，可以用于在稀疏和密集模型之间建立混合模型，具有进一步的研究意义。

Mr. TyDi: 为密集检索提供多语言评测基准

Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval

Pretrained transformer models, such as BERT and T5, have shown to be highly
effective at ad-hoc passage and document ranking. Due to inherent sequence
length limits of these models, they need to be run over a document's passages,
rather than processing the entire document sequence at once. Although several
approaches for aggregating passage-level signals have been proposed, there has
yet to be an extensive comparison of these techniques. In this work, we explore
strategies for aggregating relevance signals from a document's passages into a
final ranking score. We find that passage representation aggregation techniques
can significantly improve over techniques proposed in prior work, such as
taking the maximum passage score. We call this new approach PARADE. In
particular, PARADE can significantly improve results on collections with broad
information needs where relevance signals can be spread throughout the document
(such as TREC Robust04 and GOV2). Meanwhile, less complex aggregation
techniques may work better on collections with an information need that can
often be pinpointed to a single passage (such as TREC DL and TREC Genomics). We
also conduct efficiency analyses, and highlight several strategies for
improving transformer-based aggregation.

本文研究了如何将文档的段落级别的相关信号聚合成最终的排名分数，提出了新的 PARADE 方法，证明了它可以显著提高广泛信息需求的文集的结果，同时，也探索了提高基于 Transformer 聚合效率的方法。