Despite recent progress on computer vision and natural language processing,
developing a machine that can understand video story is still hard to achieve
due to the intrinsic difficulty of video story. Moreover, researches on how to
evaluate the degree of video understanding based on human cognitive process
have not progressed as yet. In this paper, we propose a novel video question
answering (Video QA) task, DramaQA, for a comprehensive understanding of the
video story. The DramaQA focuses on two perspectives: 1) Hierarchical QAs as an
evaluation metric based on the cognitive developmental stages of human
intelligence. 2) Character-centered video annotations to model local coherence
of the story. Our dataset is built upon the TV drama "Another Miss Oh" and it
contains 17,983 QA pairs from 23,928 various length video clips, with each QA
pair belonging to one of four difficulty levels. We provide 217,308 annotated
images with rich character-centered annotations, including visual bounding
boxes, behaviors and emotions of main characters, and coreference resolved
scripts. Additionally, we suggest Multi-level Context Matching model which
hierarchically understands character-centered representations of video to
answer questions. We release our dataset and model publicly for research
purposes, and we expect our work to provide a new perspective on video story
understanding research.

本研究提供了一种综合理解视频故事的新的视频问答（Video QA）任务，名为 DramaQA，使用层次化的问题回答和基于人类认知过程的评估标准，以及基于角色的视频注释来建模故事的局部连贯性。研究者们还提供了一个多层级上下文匹配模型来回答问题，并公开了相关数据及模型。