Despite recent improvements in abstractive summarization, most current
approaches generate summaries that are not factually consistent with the source
document, severely restricting their trust and usage in real-world
applications. Recent works have shown promising improvements in factuality
error identification using text or dependency arc entailments; however, they do
not consider the entire semantic graph simultaneously. To this end, we propose
FactGraph, a method that decomposes the document and the summary into
structured meaning representations (MR), which are more suitable for factuality
evaluation. MRs describe core semantic concepts and their relations,
aggregating the main content in both document and summary in a canonical form,
and reducing data sparsity. FactGraph encodes such graphs using a graph encoder
augmented with structure-aware adapters to capture interactions among the
concepts based on the graph connectivity, along with text representations using
an adapter-based text encoder. Experiments on different benchmarks for
evaluating factuality show that FactGraph outperforms previous approaches by up
to 15%. Furthermore, FactGraph improves performance on identifying content
verifiability errors and better captures subsentence-level factual
inconsistencies.

FactGraph 方法使用结构化的意义表示形式以及基于图的编码器和适配器来提高文本摘要的真实性。

使用语义图表示评估摘要中的事实准确性

FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations

We improve the informativeness of models for conditional text generation
using techniques from computational pragmatics. These techniques formulate
language production as a game between speakers and listeners, in which a
speaker should generate output text that a listener can use to correctly
identify the original input that the text describes. While such approaches are
widely used in cognitive science and grounded language learning, they have
received less attention for more standard language generation tasks. We
consider two pragmatic modeling methods for text generation: one where
pragmatics is imposed by information preservation, and another where pragmatics
is imposed by explicit modeling of distractors. We find that these methods
improve the performance of strong existing systems for abstractive
summarization and generation from structured meaning representations.

本文利用计算语用学中的技术，将语言生成模型的建模过程形式化为说话者和听者之间的游戏，通过两种语用建模方法，即保留信息和建模干扰项，来提高抽象概括和结构化含义表述的生成系统的性能。