This paper creates a paradigm shift with regard to the way we build neural
extractive summarization systems. Instead of following the commonly used
framework of extracting sentences individually and modeling the relationship
between sentences, we formulate the extractive summarization task as a semantic
text matching problem, in which a source document and candidate summaries will
be (extracted from the original text) matched in a semantic space. Notably,
this paradigm shift to semantic matching framework is well-grounded in our
comprehensive analysis of the inherent gap between sentence-level and
summary-level extractors based on the property of the dataset.
Besides, even instantiating the framework with a simple form of a matching
model, we have driven the state-of-the-art extractive result on CNN/DailyMail
to a new level (44.41 in ROUGE-1). Experiments on the other five datasets also
show the effectiveness of the matching framework. We believe the power of this
matching-based summarization framework has not been fully exploited. To
encourage more instantiations in the future, we have released our codes,
processed dataset, as well as generated summaries in
this https URL

通过语义文本匹配任务范式，本论文提出了一种新的神经抽取式摘要系统构建方法。经过与原有抽取式摘要方法数据集的实验对比，该方法取得了更好的抽取结果，并通过对 CNN / DailyMail 数据集的实验，创造了 44.41 的新高水平。我们相信，这种基于匹配的摘要框架的潜力还未完全被利用。

基于文本匹配的抽取式摘要

Extractive Summarization as Text Matching

Neural extractive summarization models usually employ a hierarchical encoder
for document encoding and they are trained using sentence-level labels, which
are created heuristically using rule-based methods. Training the hierarchical
encoder with these \emph{inaccurate} labels is challenging. Inspired by the
recent work on pre-training transformer sentence encoders
\cite{devlin:2018:arxiv}, we propose {\sc Hibert} (as shorthand for {\bf
HI}erachical {\bf B}idirectional {\bf E}ncoder {\bf R}epresentations from {\bf
T}ransformers) for document encoding and a method to pre-train it using
unlabeled data. We apply the pre-trained {\sc Hibert} to our summarization
model and it outperforms its randomly initialized counterpart by 1.25 ROUGE on
the CNN/Dailymail dataset and by 2.0 ROUGE on a version of New York Times
dataset. We also achieve the state-of-the-art performance on these two
datasets.

本文提出了一种名为 Hibert 的预训练模型，并将其应用于神经抽取性摘要模型中，通过使用该模型获得了 CNBC / Dailymail 数据集 1.25 ROUGE 和 New York Times 的 2.0 ROUGE， 相比于随机初始化的对应组件，性能更具优势，实现了这两个数据集的最新性能。

HIBERT：基于分层双向 Transformer 的文档级预训练技术用于文档摘要

HIBERT: Document Level Pre-training of Hierarchical Bidirectional  Transformers for Document Summarization

Coherence plays a critical role in producing a high-quality summary from a
document. In recent years, neural extractive summarization is becoming
increasingly attractive. However, most of them ignore the coherence of
summaries when extracting sentences. As an effort towards extracting coherent
summaries, we propose a neural coherence model to capture the cross-sentence
semantic and syntactic coherence patterns. The proposed neural coherence model
obviates the need for feature engineering and can be trained in an end-to-end
fashion using unlabeled data. Empirical results show that the proposed neural
coherence model can efficiently capture the cross-sentence coherence patterns.
Using the combined output of the neural coherence model and ROUGE package as
the reward, we design a reinforcement learning method to train a proposed
neural extractive summarizer which is named Reinforced Neural Extractive
Summarization (RNES) model. The RNES model learns to optimize coherence and
informative importance of the summary simultaneously. Experimental results show
that the proposed RNES outperforms existing baselines and achieves
state-of-the-art performance in term of ROUGE on CNN/Daily Mail dataset. The
qualitative evaluation indicates that summaries produced by RNES are more
coherent and readable.

本文提出了一种名为 RNES 的增强型神经汇总模型，在使用无标签数据的情况下，通过机器学习的方式训练该模型以优化汇总的连贯性和信息重要性，并通过实验证明了 RNES 的性能优于现有的基准模型，达到了 CNN/Daily  Mail 数据集的最新性能。