The task of Video Question Answering (VideoQA) consists in answering natural language questions about a video and serves as a proxy to evaluate the performance of a model in scene sequence understanding. Most methods designed for VideoQA up-to-date are end-to-end deep learning architectures which struggle at complex temporal and causal reasoning and provide limited transparency in reasoning steps. We present the HySTER: a Hybrid Spatio-Temporal Event Reasoner to reason over physical events in videos. Our model leverages the strength of deep learning methods to extract information from video frames with the reasoning capabilities and explainability of symbolic artificial intelligence in an answer set programming framework. We define a method based on general temporal, causal and physics rules which can be transferred across tasks. We apply our model to the CLEVRER dataset and demonstrate state-of-the-art results in question answering accuracy. This work sets the foundations for the incorporation of inductive logic programming in the field of VideoQA.

本文介绍HySTER：一种可以理解视频中的物理事件的混合时空事件推理器，将深度学习和符号人工智能的推理能力和说明性相结合应用于VideoQA问题，采用一种基于时间、因果和物理规则的方法，并在CLEVRER数据集上展示出了最先进的问题回答准确性结果。

HySTER: 一个混合空间时间事件推理器