What does it take to design a machine that learns to answer natural questions
about a video? A Video QA system must simultaneously understand language,
represent visual content over space-time, and iteratively transform these
representations in response to lingual content in the query, and finally
arriving at a sensible answer. While recent advances in lingual and visual
question answering have enabled sophisticated representations and neural
reasoning mechanisms, major challenges in Video QA remain on dynamic grounding
of concepts, relations and actions to support the reasoning process. Inspired
by the dual-process account of human reasoning, we design a dual process neural
architecture, which is composed of a question-guided video processing module
(System 1, fast and reactive) followed by a generic reasoning module (System 2,
slow and deliberative). System 1 is a hierarchical model that encodes visual
patterns about objects, actions and relations in space-time given the textual
cues from the question. The encoded representation is a set of high-level
visual features, which are then passed to System 2. Here multi-step inference
follows to iteratively chain visual elements as instructed by the textual
elements. The system is evaluated on the SVQA (synthetic) and TGIF-QA datasets
(real), demonstrating competitive results, with a large margin in the case of
multi-step reasoning.

本研究提出了一种基于双过程神经架构的视频问答系统，该系统能够理解语言、表示视频内容，并通过多步推理过程得出答案