Multimodal event argument role labeling (EARL), a task that assigns a role
for each event participant (object) in an image is a complex challenge. It
requires reasoning over the entire image, the depicted event, and the
interactions between various objects participating in the event. Existing
models heavily rely on high-quality event-annotated training data to understand
the event semantics and structures, and they fail to generalize to new event
types and domains. In this paper, we propose GenEARL, a training-free
generative framework that harness the power of the modern generative models to
understand event task descriptions given image contexts to perform the EARL
task. Specifically, GenEARL comprises two stages of generative prompting with a
frozen vision-language model (VLM) and a frozen large language model (LLM).
First, a generative VLM learns the semantics of the event argument roles and
generates event-centric object descriptions based on the image. Subsequently, a
LLM is prompted with the generated object descriptions with a predefined
template for EARL (i.e., assign an object with an event argument role). We show
that GenEARL outperforms the contrastive pretraining (CLIP) baseline by 9.4%
and 14.2% accuracy for zero-shot EARL on the M2E2 and SwiG datasets,
respectively. In addition, we outperform CLIP-Event by 22% precision on M2E2
dataset. The framework also allows flexible adaptation and generalization to
unseen domains.

在本文中，我们提出了 GenEARL，一种无需训练的生成框架，利用现代生成模型的能力来理解与图像背景相关的事件任务描述以执行 EARL 任务。该框架在 M2E2 和 SwiG 数据集上将对比预训练（CLIP）基准结果分别提高了 9.4％和 14.2％的零样本 EARL 准确率，并且在 M2E2 数据集上的精确度相较于 CLIP-Event 提高了 22％。该框架还允许对未见领域进行灵活的适应和推广。