Question Answering (QA) tasks requiring information from multiple documents
often rely on a retrieval model to identify relevant information for reasoning.
The retrieval model is typically trained to maximize the likelihood of the
labeled supporting evidence. However, when retrieving from large text corpora
such as Wikipedia, the correct answer can often be obtained from multiple
evidence candidates. Moreover, not all such candidates are labeled as positive
during annotation, rendering the training signal weak and noisy. This problem
is exacerbated when the questions are unanswerable or when the answers are
Boolean, since the model cannot rely on lexical overlap to make a connection
between the answer and supporting evidence. We develop a new parameterization
of set-valued retrieval that handles unanswerable queries, and we show that
marginalizing over this set during training allows a model to mitigate false
negatives in supporting evidence annotations. We test our method on two
multi-document QA datasets, IIRC and HotpotQA. On IIRC, we show that joint
modeling with marginalization improves model performance by 5.5 F1 points and
achieves a new state-of-the-art performance of 50.5 F1. We also show that
retrieval marginalization results in 4.1 QA F1 improvement over a
non-marginalized baseline on HotpotQA in the fullwiki setting.

本文提出了一种新的参数化集值检索模型，能够处理具有多个文档和无法回答的查询，通过对检索候选项进行边际化，进而降低样本标注噪声带来的影响，并在两个多文档 QA 数据集上取得了最优表现。

使用检索边际化减轻多文档问答中虚阴性情境

Mitigating False-Negative Contexts in Multi-document Question Answering  with Retrieval Marginalization

We introduce a new pretraining approach geared for multi-document language
modeling, incorporating two key ideas into the masked language modeling
self-supervised objective. First, instead of considering documents in
isolation, we pretrain over sets of multiple related documents, encouraging the
model to learn cross-document relationships. Second, we improve over recent
long-range transformers by introducing dynamic global attention that has access
to the entire input to predict masked tokens. We release CDLM (Cross-Document
Language Model), a new general language model for multi-document setting that
can be easily applied to downstream tasks. Our extensive analysis shows that
both ideas are essential for the success of CDLM, and work in synergy to set
new state-of-the-art results for several multi-text tasks. Code and models are
available at this https URL

该研究提出了一种新的预训练方法，针对多文档语言建模，包括两个关键思想：一是通过预训练多个相关文档集合，鼓励模型学习跨文档关系；二是通过引入动态全局注意力，改善最近的长距离 Transformer，以访问整个输入来预测掩码标记。 CDLM 是一个新的通用的多文档语言模型，可以轻松应用于下游任务。

CDLM: 跨文档语言建模

CDLM: Cross-Document Language Modeling

Most research in reading comprehension has focused on answering questions
based on individual documents or even single paragraphs. We introduce a neural
model which integrates and reasons relying on information spread within
documents and across multiple documents. We frame it as an inference problem on
a graph. Mentions of entities are nodes of this graph while edges encode
relations between different mentions (e.g., within- and cross-document
co-reference). Graph convolutional networks (GCNs) are applied to these graphs
and trained to perform multi-step reasoning. Our Entity-GCN method is scalable
and compact, and it achieves state-of-the-art results on a multi-document
question answering dataset, WikiHop (Welbl et al., 2018).

本研究提出了一种基于神经模型和图卷积网络的阅读理解方法，可以跨多个文档进行多步推理和信息推理，并在 WikiHop 数据集上实现了最先进的结果。