Multi-document summarization (MDS) refers to the task of summarizing the text
in multiple documents into a concise summary. The generated summary can save
the time of reading many documents by providing the important content in the
form of a few sentences. Abstractive MDS aims to generate a coherent and fluent
summary for multiple documents using natural language generation techniques. In
this paper, we consider the unsupervised abstractive MDS setting where there
are only documents with no groundtruh summaries provided, and we propose
Absformer, a new Transformer-based method for unsupervised abstractive summary
generation. Our method consists of a first step where we pretrain a
Transformer-based encoder using the masked language modeling (MLM) objective as
the pretraining task in order to cluster the documents into semantically
similar groups; and a second step where we train a Transformer-based decoder to
generate abstractive summaries for the clusters of documents. To our knowledge,
we are the first to successfully incorporate a Transformer-based model to solve
the unsupervised abstractive MDS task. We evaluate our approach using three
real-world datasets from different domains, and we demonstrate both substantial
improvements in terms of evaluation metrics over state-of-the-art
abstractive-based methods, and generalization to datasets from different
domains.

本文提出了一种新的基于 Transformer 的方法，称为 Absformer，旨在针对没有提供摘要总结的文档进行无监督的抽象式多文档摘要生成，并在现实世界的三个不同领域的数据集上进行了评估，结果表明该方法在评价指标方面取得了实质性的提高，并且具有从不同领域的数据集中泛化的优势。