This study proposes a multitask learning architecture for extractive
summarization with coherence boosting. The architecture contains an extractive
summarizer and coherent discriminator module. The coherent discriminator is
trained online on the sentence vectors of the augmented textual input, thus
improving its general ability of judging whether the input sentences are
coherent. Meanwhile, we maximize the coherent scores from the coherent
discriminator by updating the parameters of the summarizer. To make the
extractive sentences trainable in a differentiable manner, we introduce two
strategies, including pre-trained converting model (model-based) and converting
matrix (MAT-based) that merge sentence representations. Experiments show that
our proposed method significantly improves the proportion of consecutive
sentences in the extracted summaries based on their positions in the original
article (i.e., automatic sentence-level coherence metric), while the goodness
in terms of other automatic metrics (i.e., Rouge scores and BertScores) are
preserved. Human evaluation also evidences the improvement of coherence and
consistency of the extracted summaries given by our method.

提出了一种多任务学习架构，用于基于从文本中提取摘要，其中包含摘要器和连贯判别器模块，通过预训练的转换模型（基于模型）和转换矩阵（MAT-based）将句子表示合并，实现了可微分训练，并以此最大化了连贯判别器的连贯度分数，并通过人工评估证实了我们方法提高了从原始文章位置开始提取的连续句子的比例以及有效保留了其他自动评估指标。

多任务学习增强提取式摘要的连贯性

Enhancing Coherence of Extractive Summarization with Multitask Learning

Long-range semantic coherence remains a challenge in automatic language
generation and understanding. We demonstrate that large language models have
insufficiently learned the effect of distant words on next-token prediction. We
present coherence boosting, an inference procedure that increases a LM's focus
on a long context. We show the benefits of coherence boosting with pretrained
models by distributional analyses of generated ordinary text and dialog
responses. It is also found that coherence boosting with state-of-the-art
models for various zero-shot NLP tasks yields performance gains with no
additional training.

本文研究了大语言模型在长范围语义连贯性方面的问题，提出一种名为 “连贯提升” 的推理方法，能够提高语言模型对长内容的关注，通过分布分析证明了连贯提升的优点，并发现其在零样本自然语言处理任务中也能产生性能提升，而无需额外的训练。