Recent advancements in large multimodal language models have demonstrated
remarkable proficiency across a wide range of tasks. Yet, these models still
struggle with understanding the nuances of human humor through juxtaposition,
particularly when it involves nonlinear narratives that underpin many jokes and
humor cues. This paper investigates this challenge by focusing on comics with
contradictory narratives, where each comic consists of two panels that create a
humorous contradiction. We introduce the YesBut benchmark, which comprises
tasks of varying difficulty aimed at assessing AI's capabilities in recognizing
and interpreting these comics, ranging from literal content comprehension to
deep narrative reasoning. Through extensive experimentation and analysis of
recent commercial or open-sourced large (vision) language models, we assess
their capability to comprehend the complex interplay of the narrative humor
inherent in these comics. Our results show that even state-of-the-art models
still lag behind human performance on this task. Our findings offer insights
into the current limitations and potential improvements for AI in understanding
human creative expressions.

通过对具有矛盾叙事的漫画展开研究，本文引入了 YesBut 评估基准，旨在评估人工智能在识别和解释具有幽默矛盾的漫画方面的能力。通过广泛的实验和分析，我们发现即使是最先进的模型在此任务上仍然落后于人类表现，为人工智能在理解人类创造性表达方面的当前限制和潜在改进提供了见解。