Causal video question answering (QA) has garnered increasing interest, yet
existing datasets often lack depth in causal reasoning analysis. To address
this gap, we capitalize on the unique properties of cartoons and construct
CausalChaos!, a novel, challenging causal Why-QA dataset built upon the iconic
"Tom and Jerry" cartoon series. With thoughtful questions and multi-level
answers, our dataset contains much longer causal chains embedded in dynamic
interactions and visuals, at the same time principles of animation allows
animators to create well-defined, unambiguous causal relationships. These
factors allow models to solve more challenging, yet well-defined causal
relationships. We also introduce hard negative mining, including
CausalConfusion version. While models perform well, there is much room for
improvement, especially, on open-ended answers. We identify more
advanced/explicit causal relationship modeling and joint modeling of vision and
language as the immediate areas for future efforts to focus upon. Along with
the other complementary datasets, our new challenging dataset will pave the way
for these developments in the field. We will release our dataset, codes, and
models to help future efforts in this domain.

利用卡通片的独特特性构建了新的挑战性因果 Why-QA 数据集 CausalChaos!，其中包含了较长的因果链并嵌入动态交互和视觉中，模型可以通过动画原理解决更具挑战性且明确定义的因果关系问题。同时，我们的研究还指出了更先进 / 明确的因果关系建模和视觉和语言的联合建模作为未来努力的重点领域。

CausalChaos！动态视觉场景中基于较长因果链的综合因果行动问题解答数据集

CausalChaos! Dataset for Comprehensive Causal Action Question Answering  Over Longer Causal Chains Grounded in Dynamic Visual Scenes

Causal Video Question Answering (CVidQA) queries not only association or
temporal relations but also causal relations in a video. Existing question
synthesis methods pre-trained question generation (QG) systems on reading
comprehension datasets with text descriptions as inputs. However, QG models
only learn to ask association questions (e.g., ``what is someone doing...'')
and result in inferior performance due to the poor transfer of association
knowledge to CVidQA, which focuses on causal questions like ``why is someone
doing ...''. Observing this, we proposed to exploit causal knowledge to
generate question-answer pairs, and proposed a novel framework, Causal
Knowledge Extraction from Language Models (CaKE-LM), leveraging causal
commonsense knowledge from language models to tackle CVidQA. To extract
knowledge from LMs, CaKE-LM generates causal questions containing two events
with one triggering another (e.g., ``score a goal'' triggers ``soccer player
kicking ball'') by prompting LM with the action (soccer player kicking ball) to
retrieve the intention (to score a goal). CaKE-LM significantly outperforms
conventional methods by 4% to 6% of zero-shot CVidQA accuracy on NExT-QA and
Causal-VidQA datasets. We also conduct comprehensive analyses and provide key
findings for future research.

提出了一种从语言模型中提取因果知识的框架 CaKE-LM，用于解决视频因果问答问题（CVidQA），并在 NExT-QA 和 Causal-VidQA 数据集上取得了显著的优势。