Causal reasoning is fundamental to human intelligence and crucial for
effective decision-making in real-world environments. Despite recent
advancements in large vision-language models (LVLMs), their ability to
comprehend causality remains unclear. Previous work typically focuses on
commonsense causality between events and/or actions, which is insufficient for
applications like embodied agents and lacks the explicitly defined causal
graphs required for formal causal reasoning. To overcome these limitations, we
introduce a fine-grained and unified definition of causality involving
interactions between humans and/or objects. Building on the definition, we
construct a novel dataset, CELLO, consisting of 14,094 causal questions across
all four levels of causality: discovery, association, intervention, and
counterfactual. This dataset surpasses traditional commonsense causality by
including explicit causal graphs that detail the interactions between humans
and objects. Extensive experiments on CELLO reveal that current LVLMs still
struggle with causal reasoning tasks, but they can benefit significantly from
our proposed CELLO-CoT, a causally inspired chain-of-thought prompting
strategy. Both quantitative and qualitative analyses from this study provide
valuable insights for future research. Our project page is at
this https URL

通过细致而统一的因果关系定义以及涉及人类与 / 或物体之间相互作用的显式因果图，本论文构建了一个名为 CELLO 的新数据集，并表明当前的大规模视觉语言模型在因果推理任务上依然面临困难，但可以从提出的因果启发式思维提示策略 CELLO-CoT 中获得显著的收益。