Causal reasoning is fundamental to human intelligence and crucial for
effective decision-making in real-world environments. Despite recent
advancements in large vision-language models (LVLMs), their ability to
comprehend causality remains unclear. Previous work typically focuses on
commonsense causality between events and/or actions, which is insufficient for
applications like embodied agents and lacks the explicitly defined causal
graphs required for formal causal reasoning. To overcome these limitations, we
introduce a fine-grained and unified definition of causality involving
interactions between humans and/or objects. Building on the definition, we
construct a novel dataset, CELLO, consisting of 14,094 causal questions across
all four levels of causality: discovery, association, intervention, and
counterfactual. This dataset surpasses traditional commonsense causality by
including explicit causal graphs that detail the interactions between humans
and objects. Extensive experiments on CELLO reveal that current LVLMs still
struggle with causal reasoning tasks, but they can benefit significantly from
our proposed CELLO-CoT, a causally inspired chain-of-thought prompting
strategy. Both quantitative and qualitative analyses from this study provide
valuable insights for future research. Our project page is at
this https URL

通过细致而统一的因果关系定义以及涉及人类与 / 或物体之间相互作用的显式因果图，本论文构建了一个名为 CELLO 的新数据集，并表明当前的大规模视觉语言模型在因果推理任务上依然面临困难，但可以从提出的因果启发式思维提示策略 CELLO-CoT 中获得显著的收益。

CELLO: 大型视觉 - 语言模型的因果评估

CELLO: Causal Evaluation of Large Vision-Language Models

Selection bias is ubiquitous in real-world data, and can lead to misleading
results if not dealt with properly. We introduce a conditioning operation on
Structural Causal Models (SCMs) to model latent selection from a causal
perspective. We show that the conditioning operation transforms an SCM with the
presence of an explicit latent selection mechanism into an SCM without such
selection mechanism, which partially encodes the causal semantics of the
selected subpopulation according to the original SCM. Furthermore, we show that
this conditioning operation preserves the simplicity, acyclicity, and linearity
of SCMs, and commutes with marginalization. Thanks to these properties,
combined with marginalization and intervention, the conditioning operation
offers a valuable tool for conducting causal reasoning tasks within causal
models where latent details have been abstracted away. We demonstrate by
example how classical results of causal inference can be generalized to include
selection bias and how the conditioning operation helps with modeling of
real-world problems.

选择偏差在现实世界的数据中普遍存在，如果不正确处理可能导致误导性的结果。我们介绍了一种在结构性因果模型 (SCM) 上进行条件操作的方法，以从因果的角度建模潜在选择。我们展示了该条件操作将具有明确潜在选择机制存在的 SCM 转化为不具有这种选择机制的 SCM，并部分编码了根据原始 SCM 选择子群体的因果语义。此外，我们还展示了该条件操作保留了 SCM 的简易性、非循环性和线性性，并与边际化交换。凭借这些特性，结合边际化和干预，条件操作为在潜在细节被抽象掉的因果模型中进行因果推理任务提供了有价值的工具。我们通过示例演示了如何将经典的因果推断结果推广到包括选择偏差的情况，并展示了条件操作在建模现实问题方面的帮助。

利用结构因果模型进行潜在选择建模

Modeling Latent Selection with Structural Causal Models

The ability to discover abstract physical concepts and understand how they
work in the world through observing lies at the core of human intelligence. The
acquisition of this ability is based on compositionally perceiving the
environment in terms of objects and relations in an unsupervised manner. Recent
approaches learn object-centric representations and capture visually observable
concepts of objects, e.g., shape, size, and location. In this paper, we take a
step forward and try to discover and represent intrinsic physical concepts such
as mass and charge. We introduce the PHYsical Concepts Inference NEtwork
(PHYCINE), a system that infers physical concepts in different abstract levels
without supervision. The key insights underlining PHYCINE are two-fold,
commonsense knowledge emerges with prediction, and physical concepts of
different abstract levels should be reasoned in a bottom-up fashion. Empirical
evaluation demonstrates that variables inferred by our system work in
accordance with the properties of the corresponding physical concepts. We also
show that object representations containing the discovered physical concepts
variables could help achieve better performance in causal reasoning tasks,
i.e., ComPhy.

本文介绍了一个无需监督训练的系统 PHYsical Concepts Inference NEtwork (PHYCINE)，该系统可以发现和表示物理概念，包括物质和电荷，并且使用这些变量来获得比仅使用可见对象特征更好的因果推理性能。