We introduce ART, a new corpus-level autoencoding approach for training dense
retrieval models that does not require any labeled training data. Dense
retrieval is a central challenge for open-domain tasks, such as Open QA, where
state-of-the-art methods typically require large supervised datasets with
custom hard-negative mining and denoising of positive examples. ART, in
contrast, only requires access to unpaired inputs and outputs (e.g. questions
and potential answer documents). It uses a new document-retrieval autoencoding
scheme, where (1) an input question is used to retrieve a set of evidence
documents, and (2) the documents are then used to compute the probability of
reconstructing the original question. Training for retrieval based on question
reconstruction enables effective unsupervised learning of both document and
question encoders, which can be later incorporated into complete Open QA
systems without any further finetuning. Extensive experiments demonstrate that
ART obtains state-of-the-art results on multiple QA retrieval benchmarks with
only generic initialization from a pre-trained language model, removing the
need for labeled data and task-specific losses.

ART 是一种新的语料库自动编码方法，用于训练密集检索模型，通过未标注的输入和输出实现了高效的无监督学习，且在不需要标记数据和任务特定的损失的情况下，在多个 QA 检索基准测试中获得了最先进的结果。