Machine Reading Comprehension (MRC) poses a significant challenge in the
field of Natural Language Processing (NLP). While mainstream MRC methods
predominantly leverage extractive strategies using encoder-only models such as
BERT, generative approaches face the issue of out-of-control generation -- a
critical problem where answers generated are often incorrect, irrelevant, or
unfaithful to the source text. To address these limitations in generative
models for MRC, we introduce the Question-Attended Span Extraction (QASE)
module. Integrated during the fine-tuning phase of pre-trained generative
language models (PLMs), QASE significantly enhances their performance, allowing
them to surpass the extractive capabilities of advanced Large Language Models
(LLMs) such as GPT-4. Notably, these gains in performance do not come with an
increase in computational demands. The efficacy of the QASE module has been
rigorously tested across various datasets, consistently achieving or even
surpassing state-of-the-art (SOTA) results.

在自然语言处理领域，机器阅读理解（MRC）存在着重大挑战。尽管主流的 MRC 方法主要使用仅编码器模型（如 BERT）利用抽取策略，但生成式方法面临着超出控制的生成问题 —— 这是一个关键问题，生成的答案经常是不正确、不相关或与源文本不一致。为了解决生成式 MRC 模型的这些局限性，我们引入了 Question-Attended Span Extraction（QASE）模块。在经过预训练的生成式语言模型（PLM）的微调阶段集成 QASE 模块，显著提高了它们的性能，使它们能够超越像 GPT-4 这样的先进大型语言模型（LLMs）的抽取能力。值得注意的是，这种性能提升并没有增加计算需求。QASE 模块的有效性已经在各种数据集上进行了严格的测试，始终实现甚至超越最先进的结果（SOTA）。

用问题关注的段落抽取增强预设的生成式语言模型在机器阅读理解中的应用

Enhancing Pre-Trained Generative Language Models with Question Attended  Span Extraction on Machine Reading Comprehension

We present NEWSROOM, a summarization dataset of 1.3 million articles and
summaries written by authors and editors in newsrooms of 38 major news
publications. Extracted from search and social media metadata between 1998 and
2017, these high-quality summaries demonstrate high diversity of summarization
styles. In particular, the summaries combine abstractive and extractive
strategies, borrowing words and phrases from articles at varying rates. We
analyze the extraction strategies used in NEWSROOM summaries against other
datasets to quantify the diversity and difficulty of our new data, and train
existing methods on the data to evaluate its utility and challenges.

利用搜索和社交媒体元数据，我们构建了 1.3 百万篇文章及其摘要的 NEWSROOM 数据集，包含不同风格的抽象和抽取策略，为评估该数据集的效用和挑战，我们对该数据集进行了分析和训练现有方法。