The performance of automatic summarization models has improved dramatically
in recent years. Yet, there is still a gap in meeting specific information
needs of users in real-world scenarios, particularly when a targeted summary is
sought, such as in the useful aspect-based summarization setting targeted in
this paper. Previous datasets and studies for this setting have predominantly
concentrated on a limited set of pre-defined aspects, focused solely on single
document inputs, or relied on synthetic data. To advance research on more
realistic scenarios, we introduce OpenAsp, a benchmark for multi-document
\textit{open} aspect-based summarization. This benchmark is created using a
novel and cost-effective annotation protocol, by which an open aspect dataset
is derived from existing generic multi-document summarization datasets. We
analyze the properties of OpenAsp showcasing its high-quality content. Further,
we show that the realistic open-aspect setting realized in OpenAsp poses a
challenge for current state-of-the-art summarization models, as well as for
large language models.

近年来，自动摘要模型的性能有了显著提升，然而在满足实际场景中用户的具体信息需求方面仍存在差距，尤其是在本文中所针对的有用的基于方面的摘要设置中。为了推动更加现实的场景研究，我们引入了 OpenAsp，一个用于多文档开放方面的摘要的基准。该基准使用一种新颖且具有成本效益的注释协议来创建，通过该协议从现有的通用多文档摘要数据集中提取开放方面数据集。我们分析了 OpenAsp 的特性，展示了其高质量的内容，并且证明了 OpenAsp 中实现的现实开放方面设置对于当前最先进的摘要模型和大型语言模型来说是一个挑战。