Event coreference resolution (ECR) is the task of determining whether
distinct mentions of events within a multi-document corpus are actually linked
to the same underlying occurrence. Images of the events can help facilitate
resolution when language is ambiguous. Here, we propose a multimodal
cross-document event coreference resolution method that integrates visual and
textual cues with a simple linear map between vision and language models. As
existing ECR benchmark datasets rarely provide images for all event mentions,
we augment the popular ECB+ dataset with event-centric images scraped from the
internet and generated using image diffusion models. We establish three methods
that incorporate images and text for coreference: 1) a standard fused model
with finetuning, 2) a novel linear mapping method without finetuning and 3) an
ensembling approach based on splitting mention pairs by semantic and
discourse-level difficulty. We evaluate on 2 datasets: the augmented ECB+, and
AIDA Phase 1. Our ensemble systems using cross-modal linear mapping establish
an upper limit (91.9 CoNLL F1) on ECB+ ECR performance given the preprocessing
assumptions used, and establish a novel baseline on AIDA Phase 1. Our results
demonstrate the utility of multimodal information in ECR for certain
challenging coreference problems, and highlight a need for more multimodal
resources in the coreference resolution space.

提出了一种融合视觉和文本线性映射的多模态跨文档事件共指解析方法，通过对事件中心图像的集成，对核心指称进行融合模型、线性映射和集合方法进行处理，实现了对事件共指的跨模态线性映射，为核心指称解析领域提供了多模态信息的实用性和需要更多多模态资源的提示。