Pre-trained sentence representations are crucial for identifying significant
sentences in unsupervised document extractive summarization. However, the
traditional two-step paradigm of pre-training and sentence-ranking, creates a
gap due to differing optimization objectives. To address this issue, we argue
that utilizing pre-trained embeddings derived from a process specifically
designed to optimize cohensive and distinctive sentence representations helps
rank significant sentences. To do so, we propose a novel graph pre-training
auto-encoder to obtain sentence embeddings by explicitly modelling
intra-sentential distinctive features and inter-sentential cohesive features
through sentence-word bipartite graphs. These pre-trained sentence
representations are then utilized in a graph-based ranking algorithm for
unsupervised summarization. Our method produces predominant performance for
unsupervised summarization frameworks by providing summary-worthy sentence
representations. It surpasses heavy BERT- or RoBERTa-based sentence
representations in downstream tasks.

利用预训练句子表示解决自动摘要中的关键句子识别问题，通过图预训练自编码器获得句子嵌入，并应用基于图的排名算法进行无监督摘要生成，表现优于基于 BERT 或 RoBERTa 的句子表示方法。

使用双分图预训练进行无监督的图卷积自编码器抽取式摘要

Bipartite Graph Pre-training for Unsupervised Extractive Summarization  with Graph Convolutional Auto-Encoders

Single document summarization has enjoyed renewed interests in recent years
thanks to the popularity of neural network models and the availability of
large-scale datasets. In this paper we develop an unsupervised approach arguing
that it is unrealistic to expect large-scale and high-quality training data to
be available or created for different types of summaries, domains, or
languages. We revisit a popular graph-based ranking algorithm and modify how
node (aka sentence) centrality is computed in two ways: (a)~we employ BERT, a
state-of-the-art neural representation learning model to better capture
sentential meaning and (b)~we build graphs with directed edges arguing that the
contribution of any two nodes to their respective centrality is influenced by
their relative position in a document. Experimental results on three news
summarization datasets representative of different languages and writing styles
show that our approach outperforms strong baselines by a wide margin.

本文提出了一种针对新闻摘要自动提取的无监督方法，该方法采用神经网络模型并结合基于图论的排名算法，使用 BERT 模型来更好地捕获句子意义，显著超过强基线的效果。