The summarization literature focuses on the summarization of news articles.
The news articles in the CNN-DailyMail are relatively short documents with
about 30 sentences per document on average. We introduce SciBERTSUM, our
summarization framework designed for the summarization of long documents like
scientific papers with more than 500 sentences. SciBERTSUM extends BERTSUM to
long documents by 1) adding a section embedding layer to include section
information in the sentence vector and 2) applying a sparse attention mechanism
where each sentences will attend locally to nearby sentences and only a small
number of sentences attend globally to all other sentences. We used slides
generated by the authors of scientific papers as reference summaries since they
contain the technical details from the paper. The results show the superiority
of our model in terms of ROUGE scores.

SciBERTSUM 是一种针对科学论文的摘要框架，通过增加节嵌入层和应用稀疏注意力机制，在长文本（超过 500 句）中表现出更好的 ROUGE 分数。

SciBERTSUM: 面向科学文档的提取式摘要

SciBERTSUM: Extractive Summarization for Scientific Documents

Summarization of speech is a difficult problem due to the spontaneity of the
flow, disfluencies, and other issues that are not usually encountered in
written texts. Our work presents the first application of the BERTSum model to
conversational language. We generate abstractive summaries of narrated
instructional videos across a wide variety of topics, from gardening and
cooking to software configuration and sports. In order to enrich the
vocabulary, we use transfer learning and pretrain the model on a few large
cross-domain datasets in both written and spoken English. We also do
preprocessing of transcripts to restore sentence segmentation and punctuation
in the output of an ASR system. The results are evaluated with ROUGE and
Content-F1 scoring for the How2 and WikiHow datasets. We engage human judges to
score a set of summaries randomly selected from a dataset curated from
HowTo100M and YouTube. Based on blind evaluation, we achieve a level of textual
fluency and utility close to that of summaries written by human content
creators. The model beats current SOTA when applied to WikiHow articles that
vary widely in style and topic, while showing no performance regression on the
canonical CNN/DailyMail dataset. Due to the high generalizability of the model
across different styles and domains, it has great potential to improve
accessibility and discoverability of internet content. We envision this
integrated as a feature in intelligent virtual assistants, enabling them to
summarize both written and spoken instructional content upon request.

本研究使用 BERTSum 模型对按主题分类的教学视频进行抽象摘要概括，通过多个英文语境下的数据集预训练模型，使用 ROUGE 和 Content-F1 评分进行结果评估，同时进行人工盲评，结果显示本方法在 WikiHow 数据集中具有较高的效果及概括性能力。

使用 BERT 进行口头和书面指令的抽象摘要

Abstractive Summarization of Spoken and Written Instructions with BERT

BERT, a pre-trained Transformer model, has achieved ground-breaking
performance on multiple NLP tasks. In this paper, we describe BERTSUM, a simple
variant of BERT, for extractive summarization. Our system is the state of the
art on the CNN/Dailymail dataset, outperforming the previous best-performed
system by 1.65 on ROUGE-L. The codes to reproduce our results are available at
this https URL

本文提出了 BERTSUM，即 BERT 的摘要变体，通过在 CNN / Dailymail 数据集上的评估证明其在提取性摘要方面超越了现有的最佳系统。