Causality knowledge is crucial for many artificial intelligence systems.
Conventional textual-based causality knowledge acquisition methods typically
require laborious and expensive human annotations. As a result, their scale is
often limited. Moreover, as no context is provided during the annotation, the
resulting causality knowledge records (e.g., ConceptNet) typically do not take
the context into consideration. To explore a more scalable way of acquiring
causality knowledge, in this paper, we jump out of the textual domain and
investigate the possibility of learning contextual causality from the visual
signal. Compared with pure text-based approaches, learning causality from the
visual signal has the following advantages: (1) Causality knowledge belongs to
the commonsense knowledge, which is rarely expressed in the text but rich in
videos; (2) Most events in the video are naturally time-ordered, which provides
a rich resource for us to mine causality knowledge from; (3) All the objects in
the video can be used as context to study the contextual property of causal
relations. In detail, we first propose a high-quality dataset Vis-Causal and
then conduct experiments to demonstrate that with good language and visual
representation models as well as enough training signals, it is possible to
automatically discover meaningful causal knowledge from the videos. Further
analysis also shows that the contextual property of causal relations indeed
exists, taking which into consideration might be crucial if we want to use the
causality knowledge in real applications, and the visual signal could serve as
a good resource for learning such contextual causality.

本文探讨了从视觉信号中学习上下文因果关系的可能性，提出了高质量数据集 Vis-Causal，并展示了通过好的语言和视觉表征模型和充足的训练信号，可以从视频中自动发现有意义的因果知识。