Multi-document summarization entails producing concise synopses of
collections of inputs. For some applications, the synopsis should accurately
\emph{synthesize} inputs with respect to a key property or aspect. For example,
a synopsis of film reviews all written about a particular movie should reflect
the average critic consensus. As a more consequential example, consider
narrative summaries that accompany biomedical \emph{systematic reviews} of
clinical trial results. These narratives should fairly summarize the
potentially conflicting results from individual trials.
In this paper we ask: To what extent do modern multi-document summarization
models implicitly perform this type of synthesis? To assess this we perform a
suite of experiments that probe the degree to which conditional generation
models trained for summarization using standard methods yield outputs that
appropriately synthesize inputs. We find that existing models do partially
perform synthesis, but do so imperfectly. In particular, they are
over-sensitive to changes in input ordering and under-sensitive to changes in
input compositions (e.g., the ratio of positive to negative movie reviews). We
propose a simple, general method for improving model synthesis capabilities by
generating an explicitly diverse set of candidate outputs, and then selecting
from these the string best aligned with the expected aggregate measure for the
inputs, or \emph{abstaining} when the model produces no good candidate. This
approach improves model synthesis performance. We hope highlighting the need
for synthesis (in some summarization settings), motivates further research into
multi-document summarization methods and learning objectives that explicitly
account for the need to synthesize.

本文旨在评估现代多文档摘要模型在混合输入时是否能够适当地综合输出，并提出了一种改进模型综合能力的方法，即通过生成多样的候选输出并在这些中选择与期望聚合度量最能匹配的字符串或在模型产生的候选项不好时选择回避。