Generative models have recently started to outperform extractive models in
Open Domain Question Answering, largely by leveraging their decoder to attend
over multiple encoded passages and combining their information. However,
generative models tend to be larger than extractive models due to the need for
a decoder, run slower during inference due to auto-regressive decoder beam
search, and their generated output often suffers from hallucinations. We
propose to extend transformer encoders with the ability to fuse information
from multiple passages, using global representation to provide cross-sample
attention over all tokens across samples. Furthermore, we propose an
alternative answer span probability calculation to better aggregate answer
scores in the global space of all samples. Using our proposed method, we
outperform the current state-of-the-art method by $2.5$ Exact Match score on
the Natural Question dataset while using only $25\%$ of parameters and $35\%$
of the latency during inference, and $4.4$ Exact Match on WebQuestions dataset.
When coupled with synthetic data augmentation, we outperform larger models on
the TriviaQA dataset as well. The latency and parameter savings of our method
make it particularly attractive for open-domain question answering, as these
models are often compute-intensive.

通过扩展 Transformer 编码器的能力以融合信息，使用全局表示跨样本注意所有令牌，并提出了一种更好的答案跨度概率计算方法，我们使用与综合数据增强配对的方法，在推理期间只使用 25％的参数和 35％的延迟，在自然问题数据集上的精确匹配分数达到了当前技术水平的 2.5 和 WebQuestions 数据集的 4.4。这种方法的延迟和参数节省特别适用于计算密集型应用，例如开放域问题回答。