Factual inconsistency poses a significant hurdle for the commercial
deployment of abstractive summarizers. Under this Large Language Model (LLM)
era, this work focuses around two important questions: what is the best way to
leverage LLM for factual inconsistency detection, and how could we distill a
smaller LLM with both high efficiency and efficacy? Three zero-shot paradigms
are firstly proposed and evaluated across five diverse datasets: direct
inference on the entire summary or each summary window; entity verification
through question generation and answering. Experiments suggest that LLM itself
is capable to resolve this task train-free under the proper paradigm design,
surpassing strong trained baselines by 2.8% on average. To further promote
practical utility, we then propose training strategies aimed at distilling
smaller open-source LLM that learns to score the entire summary at once with
high accuracy, which outperforms the zero-shot approaches by much larger LLM,
serving as an effective and efficient ready-to-use scorer.

通过零样本策略，本研究提出并评估了三种方式来解决实际的矛盾检测问题，并研究了如何精简高效且功效强大的大型语言模型。实验结果表明，适当设计的范式可以使大型语言模型在无需训练的情况下解决此问题，平均超越强训练基线 2.8％。为了进一步提高实用性，我们提出了训练策略，旨在通过高准确性一次对整个摘要进行评分的方式来精简开源大型语言模型，其效果优于较大的零样本大型语言模型，成为一种有效且高效的可即用得分器。

摘要中事实不一致的识别：向大型语言模型的有效利用迈进

Identifying Factual Inconsistency in Summaries: Towards Effective  Utilization of Large Language Model

Despite the success of recent abstractive summarizers on automatic evaluation
metrics, the generated summaries still present factual inconsistencies with the
source document. In this paper, we focus on entity-level factual inconsistency,
i.e. reducing the mismatched entities between the generated summaries and the
source documents. We therefore propose a novel entity-based SpanCopy mechanism,
and explore its extension with a Global Relevance component. Experiment results
on four summarization datasets show that SpanCopy can effectively improve the
entity-level factual consistency with essentially no change in the word-level
and entity-level saliency. The code is available at
this https URL

本文提出了一种基于实体级的 SpanCopy 机制和全局相关性组件的方法，用于降低自动生成摘要与源文件之间的事实矛盾，实验证明此方法有效，不改变词级和实体级显著性。

基于实体的 SpanCopy 用于提高抽象摘要的事实一致性

Entity-based SpanCopy for Abstractive Summarization to Improve the Factual Consistency

We propose a new approach to generate multiple variants of the target summary
with diverse content and varying lengths, then score and select admissible ones
according to users' needs. Abstractive summarizers trained on single reference
summaries may struggle to produce outputs that achieve multiple desirable
properties, i.e., capturing the most important information, being faithful to
the original, grammatical and fluent. In this paper, we propose a two-staged
strategy to generate a diverse set of candidate summaries from the source text
in stage one, then score and select admissible ones in stage two. Importantly,
our generator gives a precise control over the length of the summary, which is
especially well-suited when space is limited. Our selectors are designed to
predict the optimal summary length and put special emphasis on faithfulness to
the original text. Both stages can be effectively trained, optimized and
evaluated. Our experiments on benchmark summarization datasets suggest that
this paradigm can achieve state-of-the-art performance.

本研究提出一种新方法，通过生成多个变体的目标摘要，这些摘要具有不同的内容和不同的长度，并根据用户的需要进行评分和选择，该方法可用于控制摘要长度，从而实现最佳的信息概括，保证真实性并确保流畅。

一种新的从容提取摘要的方法

A New Approach to Overgenerating and Scoring Abstractive Summaries

Fusing sentences containing disparate content is a remarkable human ability
that helps create informative and succinct summaries. Such a simple task for
humans has remained challenging for modern abstractive summarizers,
substantially restricting their applicability in real-world scenarios. In this
paper, we present an investigation into fusing sentences drawn from a document
by introducing the notion of points of correspondence, which are cohesive
devices that tie any two sentences together into a coherent text. The types of
points of correspondence are delineated by text cohesion theory, covering
pronominal and nominal referencing, repetition and beyond. We create a dataset
containing the documents, source and fusion sentences, and human annotations of
points of correspondence between sentences. Our dataset bridges the gap between
coreference resolution and summarization. It is publicly shared to serve as a
basis for future work to measure the success of sentence fusion systems.
(this https URL)

本文介绍了一种利用文本连贯性理论中的对应点将文档中的句子融合成一篇连贯文本的方法，并创建了数据集以促进此技术的发展和测量。