Event-Level Video Question Answering (EVQA) requires complex reasoning across
video events to obtain the visual information needed to provide optimal
answers. However, despite significant progress in model performance, few
studies have focused on using the explicit semantic connections between the
question and visual information especially at the event level. There is need
for using such semantic connections to facilitate complex reasoning across
video frames. Therefore, we propose a semantic-aware dynamic
retrospective-prospective reasoning approach for video-based question
answering. Specifically, we explicitly use the Semantic Role Labeling (SRL)
structure of the question in the dynamic reasoning process where we decide to
move to the next frame based on which part of the SRL structure (agent, verb,
patient, etc.) of the question is being focused on. We conduct experiments on a
benchmark EVQA dataset - TrafficQA. Results show that our proposed approach
achieves superior performance compared to previous state-of-the-art models. Our
code will be made publicly available for research use.

本文提出了一种语义感知的动态回顾 - 前瞻推理方法，用于视频问答，通过使用问题的语义角色标注（SRL）结构，实现了基于语义连接的复杂推理，在 TrafficQA 数据集上实验结果显示，相较于之前的最先进模型，该方法实现了卓越的性能提升。