We propose a novel method for applying Transformer models to extractive
question answering (QA) tasks. Recently, pretrained generative
sequence-to-sequence (seq2seq) models have achieved great success in question
answering. Contributing to the success of these models are internal attention
mechanisms such as cross-attention. We propose a simple strategy to obtain an
extractive answer span from the generative model by leveraging the decoder
cross-attention patterns. Viewing cross-attention as an architectural prior, we
apply joint training to further improve QA performance. Empirical results show
that on open-domain question answering datasets like NaturalQuestions and
TriviaQA, our method approaches state-of-the-art performance on both generative
and extractive inference, all while using much fewer parameters. Furthermore,
this strategy allows us to perform hallucination-free inference while
conferring significant improvements to the model's ability to rerank relevant
passages.

本文提出了一种新的方法，利用 Transformer 模型进行抽取式问答。该方法利用预训练的生成式模型，并结合内部 cross-attention 机制实现了良好的表现，同时使用较少的参数，使模型能够进行幻觉式推断，并提高模型重排相关段落的能力。