Event extraction has attracted much attention in recent years due to its
potential for many applications. However, recent studies observe some
evaluation challenges, suggesting that reported scores might not reflect the
true performance. In this work, we first identify and discuss these evaluation
challenges, including the unfair comparisons resulting from different
assumptions about data or different data preprocessing steps, the
incompleteness of the current evaluation framework leading to potential dataset
bias or data split bias, and low reproducibility of prior studies. To address
these challenges, we propose TextEE, a standardized, fair, and reproducible
benchmark for event extraction. TextEE contains standardized data preprocessing
scripts and splits for more than ten datasets across different domains. In
addition, we aggregate and re-implement over ten event extraction approaches
published in recent years and conduct a comprehensive reevaluation. Finally, we
explore the capability of large language models in event extraction and discuss
some future challenges. We expect TextEE will serve as a reliable benchmark for
event extraction, facilitating future research in the field.

本文讨论和解决事件提取评估中的挑战，并提出了 TextEE 作为一个标准化、公平和可重现的事件提取基准，包含了多个领域的标准化数据预处理脚本和数据集切分，重新评估了多个事件提取方法，并探索了大型语言模型在事件提取中的能力和未来挑战。