We introduce MultiMUC, the first multilingual parallel corpus for template
filling, comprising translations of the classic MUC-4 template filling
benchmark into five languages: Arabic, Chinese, Farsi, Korean, and Russian. We
obtain automatic translations from a strong multilingual machine translation
system and manually project the original English annotations into each target
language. For all languages, we also provide human translations for sentences
in the dev and test splits that contain annotated template arguments. Finally,
we present baselines on MultiMUC both with state-of-the-art template filling
models and with ChatGPT.

我们介绍了 MultiMUC，这是第一个用于模板填充的多语言平行语料库，包括对经典 MUC-4 模板填充基准的五种语言（阿拉伯语、中文、波斯语、韩语和俄语）的翻译。我们从一个强大的多语言机器翻译系统获得自动翻译，并手工将原始的英文注释投射到每个目标语言中。对于所有语言，我们还提供了包含注释模板参数的开发和测试分裂中的句子的人工翻译。最后，我们使用 MultiMUC 展示了基于最新模板填充模型和 ChatGPT 的基准性能。

MultiMUC：MUC-4 上的多语言模板填充

MultiMUC: Multilingual Template Filling on MUC-4

As information extraction (IE) systems have grown more capable at
whole-document extraction, the classic task of \emph{template filling} has seen
renewed interest as a benchmark for evaluating them. In this position paper, we
call into question the suitability of template filling for this purpose. We
argue that the task demands definitive answers to thorny questions of
\emph{event individuation} -- the problem of distinguishing distinct events --
about which even human experts disagree. We show through annotation studies and
error analysis that this raises concerns about the usefulness of template
filling evaluation metrics, the quality of datasets for the task, and the
ability of models to learn it. Finally, we consider possible solutions.

本文讨论信息抽取系统区分事件过程中存在的问题，进一步探讨事件模板填充在该问题中的适用性，通过反思评估度量，数据集质量以及模型学习能力等方面，提出了解决措施。