This paper introduces OARelatedWork, the first large-scale multi-document
summarization dataset for related work generation containing whole related work
sections and full-texts of cited papers. The dataset includes 94 450 papers and
5 824 689 unique referenced papers. It was designed for the task of
automatically generating related work to shift the field toward generating
entire related work sections from all available content instead of generating
parts of related work sections from abstracts only, which is the current
mainstream in this field for abstractive approaches. We show that the estimated
upper bound for extractive summarization increases by 217% in the ROUGE-2
score, when using full content instead of abstracts. Furthermore, we show the
benefits of full content data on naive, oracle, traditional, and
transformer-based baselines. Long outputs, such as related work sections, pose
challenges for automatic evaluation metrics like BERTScore due to their limited
input length. We tackle this issue by proposing and evaluating a meta-metric
using BERTScore. Despite operating on smaller blocks, we show this meta-metric
correlates with human judgment, comparably to the original BERTScore.

该论文介绍了 OARelatedWork，这是第一个大规模的多文档摘要数据集，用于相关工作生成，包含整个相关工作部分和引用论文的全文。该数据集包括 94450 篇论文和 5824689 篇独特引用的论文。它旨在自动生成相关工作，将该领域从仅从摘要生成相关工作部分的抽象方法转向从所有可用内容生成整个相关工作部分的方式。当使用全文而不是摘要时，我们发现提取式摘要的 ROUGE-2 得分上限增加 217%。此外，我们展示了全文数据对于基线模型（如 naive、oracle、traditional 和基于 transformer 的模型）的好处。长文本输出（如相关工作部分）对于像 BERTScore 这样的自动评估指标会带来挑战，因为其输入长度有限。我们通过提出和评估使用 BERTScore 的元度量来解决这个问题。尽管操作的是较小的块，但我们证明这个元度量与人类判断具有相当的相关性，与原始的 BERTScore 相当。

OARelatedWork：一种来自开放获取资源的大规模相关工作段落全文数据集

OARelatedWork: A Large-Scale Dataset of Related Work Sections with  Full-texts from Open Access Sources

To convince readers of the novelty of their research paper, authors must
perform a literature review and compose a coherent story that connects and
relates prior works to the current work. This challenging nature of literature
review writing makes automatic related work generation (RWG) academically and
computationally interesting, and also makes it an excellent test bed for
examining the capability of SOTA natural language processing (NLP) models.
Since the initial proposal of the RWG task, its popularity has waxed and waned,
following the capabilities of mainstream NLP approaches. In this work, we
survey the zoo of RWG historical works, summarizing the key approaches and task
definitions and discussing the ongoing challenges of RWG.

文献综述、相关工作生成、自然语言处理、NLP 模型、挑战

Related Work and Citation Text Generation: A Survey

Academic research is an exploratory activity to discover new solutions to
problems. By this nature, academic research works perform literature reviews to
distinguish their novelties from prior work. In natural language processing,
this literature review is usually conducted under the "Related Work" section.
The task of related work generation aims to automatically generate the related
work section given the rest of the research paper and a list of papers to cite.
Prior work on this task has focused on the sentence as the basic unit of
generation, neglecting the fact that related work sections consist of variable
length text fragments derived from different information sources. As a first
step toward a linguistically-motivated related work generation framework, we
present a Citation Oriented Related Work Annotation (CORWA) dataset that labels
different types of citation text fragments from different information sources.
We train a strong baseline model that automatically tags the CORWA labels on
massive unlabeled related work section texts. We further suggest a novel
framework for human-in-the-loop, iterative, abstractive related work
generation.

本文介绍了一个基于语言学的相关工作自动生成框架，包括引入一个标注数据集 CORWA 和训练一个基于此标注数据集的强基线模型，并建议了一种新颖的循环抽象与人工参与的相关工作生成框架。