Tasks involving text generation based on multiple input texts, such as
multi-document summarization, long-form question answering and contemporary
dialogue applications, challenge models for their ability to properly
consolidate partly-overlapping multi-text information. However, these tasks
entangle the consolidation phase with the often subjective and ill-defined
content selection requirement, impeding proper assessment of models'
consolidation capabilities. In this paper, we suggest revisiting the sentence
union generation task as an effective well-defined testbed for assessing text
consolidation capabilities, decoupling the consolidation challenge from
subjective content selection. To support research on this task, we present
refined annotation methodology and tools for crowdsourcing sentence union,
create the largest union dataset to date and provide an analysis of its rich
coverage of various consolidation aspects. We then propose a comprehensive
evaluation protocol for union generation, including both human and automatic
evaluation. Finally, as baselines, we evaluate state-of-the-art language models
on the task, along with a detailed analysis of their capacity to address
multi-text consolidation challenges and their limitations.

本文提出将句子联合生成任务作为一种有效的明确定义的测试基准，以评估文本合并能力，消除了主观内容选择的影响。针对该任务，我们提出了一套细化的注释方法和众包工具，创建了迄今最大的联合数据集，并提供了多种合并方面的丰富分析。最后，我们对最先进的语言模型进行了基线评估，并对它们解决多文本合并挑战的能力及其局限性进行了详细分析。