Recent advances in Artificial Intelligence and deep learning have revived the interest in studying the gap between the reasoning capabilities of humans and machines. In this ongoing work, we introduce CRAFT, a new visual question answering dataset that requires causal reasoning about physical forces and object interactions. It contains 38K video and question pairs that are generated from 3K videos from 10 different virtual environments, containing different number of objects in motion that interact with each other. Two question categories from CRAFT include previously studied descriptive and counterfactual questions. Besides, inspired by the theory of force dynamics from the field of human cognitive psychology, we introduce new question categories that involve understanding the intentions of objects through the notions of cause, enable, and prevent. Our preliminary results demonstrate that even though these tasks are very intuitive for humans, the implemented baselines could not cope with the underlying challenges.

该研究引入了CRAFT数据集，该数据集包含来自20个虚拟环境的10K个视频，涉及各种互相作用的对象。通过描述性，反事实和因果问题的回答，测试了CRAFT的易用性。尽管问题对人类来说很容易，但测试的基准模型，包括现有的最先进方法，仍然无法解决我们提供的挑战。通过认知语言学中的力动态理论启发，引入涉及通过原因、使能和防止等观念理解对象之间因果相互作用的新因果问题类别。

CRAFT：关于力和相互作用因果推理的基准