We present the task of Spatio-Temporal Video Question Answering, which
requires intelligent systems to simultaneously retrieve relevant moments and
detect referenced visual concepts (people and objects) to answer natural
language questions about videos. We first augment the TVQA dataset with 310.8K
bounding boxes, linking depicted objects to visual concepts in questions and
answers. We name this augmented version as TVQA+. We then propose
Spatio-Temporal Answerer with Grounded Evidence (STAGE), a unified framework
that grounds evidence in both spatial and temporal domains to answer questions
about videos. Comprehensive experiments and analyses demonstrate the
effectiveness of our framework and how the rich annotations in our TVQA+
dataset can contribute to the question answering task. Moreover, by performing
this joint task, our model is able to produce insightful and interpretable
spatio-temporal attention visualizations. Dataset and code are publicly
available at: http: //tvqa.cs.unc.edu, this https URL

该研究针对视频问答这一任务，提出增加 bounding boxes 数据集，以此为基础构建了 STAGE 框架，在空间和时间域上对视频进行处理，以便回答关于视频的自然语言问题，并展示了实验结果和可视化。