As a foundational component of cognitive intelligence, theory of mind (ToM)
can make AI more closely resemble human thought processes, thereby enhancing
their interaction and collaboration with human. In particular, it can
significantly improve a model's comprehension of videos in complex scenes.
However, current video question answer (VideoQA) datasets focus on studying
causal reasoning within events few of them genuinely incorporating human ToM.
Consequently, there is a lack of development in ToM reasoning tasks within the
area of VideoQA. This paper presents BDIQA, the first benchmark to explore the
cognitive reasoning capabilities of VideoQA models in the context of ToM. BDIQA
is inspired by the cognitive development of children's ToM and addresses the
current deficiencies in machine ToM within datasets and tasks. Specifically, it
offers tasks at two difficulty levels, assessing Belief, Desire and Intention
(BDI) reasoning in both simple and complex scenarios. We conduct evaluations on
several mainstream methods of VideoQA and diagnose their capabilities with zero
shot, few shot and supervised learning. We find that the performance of
pre-trained models on cognitive reasoning tasks remains unsatisfactory. To
counter this challenge, we undertake thorough analysis and experimentation,
ultimately presenting two guidelines to enhance cognitive reasoning derived
from ablation analysis.

本文介绍了 BDIQA，这是第一个探索 ToM 理论在 VideoQA 模型中认知推理能力的基准测试集，旨在填补当前 VideoQA 数据集与任务中机器 ToM 方面的不足，并通过实验得出了提高认知推理能力的两项指导原则。