Understanding the nature of high-quality summaries is crucial to further
improve the performance of multi-document summarization. We propose an approach
to characterize human-written summaries using partial information
decomposition, which decomposes the mutual information provided by all source
documents into union, redundancy, synergy, and unique information. Our
empirical analysis on different MDS datasets shows that there is a direct
dependency between the number of sources and their contribution to the summary.

通过部分信息分解的方法，我们对人工撰写的摘要进行了特征化，发现源文件的数量与其对摘要的贡献存在直接依赖关系。

哪些信息起作用？通过部分信息分解解析人类编写的多文档摘要

Which Information Matters? Dissecting Human-written Multi-document  Summaries with Partial Information Decomposition

The task of multi-document summarization (MDS) aims at models that, given
multiple documents as input, are able to generate a summary that combines
disperse information, originally spread across these documents. Accordingly, it
is expected that both reference summaries in MDS datasets, as well as system
summaries, would indeed be based on such dispersed information. In this paper,
we argue for quantifying and assessing this expectation. To that end, we
propose an automated measure for evaluating the degree to which a summary is
``disperse'', in the sense of the number of source documents needed to cover
its content. We apply our measure to empirically analyze several popular MDS
datasets, with respect to their reference summaries, as well as the output of
state-of-the-art systems. Our results show that certain MDS datasets barely
require combining information from multiple documents, where a single document
often covers the full summary content. Overall, we advocate using our metric
for assessing and improving the degree to which summarization datasets require
combining multi-document information, and similarly how summarization models
actually meet this challenge. Our code is available in
this https URL.

本文介绍一种用于评估多文档摘要生成模型及数据集中数据分散程度的自动化度量方法，并检验几个流行的多文档摘要数据集，结果表明某些数据集没有利用多文档信息，提出应用该度量方法评估数据集和模型的性能。