The main challenge in video question answering (VideoQA) is to capture and
understand the complex spatial and temporal relations between objects based on
given questions. Existing graph-based methods for VideoQA usually ignore
keywords in questions and employ a simple graph to aggregate features without
considering relative relations between objects, which may lead to inferior
performance. In this paper, we propose a Keyword-aware Relative Spatio-Temporal
(KRST) graph network for VideoQA. First, to make question features aware of
keywords, we employ an attention mechanism to assign high weights to keywords
during question encoding. The keyword-aware question features are then used to
guide video graph construction. Second, because relations are relative, we
integrate the relative relation modeling to better capture the spatio-temporal
dynamics among object nodes. Moreover, we disentangle the spatio-temporal
reasoning into an object-level spatial graph and a frame-level temporal graph,
which reduces the impact of spatial and temporal relation reasoning on each
other. Extensive experiments on the TGIF-QA, MSVD-QA and MSRVTT-QA datasets
demonstrate the superiority of our KRST over multiple state-of-the-art methods.

该论文提出了一种关键词感知的相对时空图网络（KRST）用于视频问答，通过在问题编码过程中使用注意机制让问题特征对关键词敏感，指导视频图构建，并整合了相对关系建模以更好地捕捉物体节点之间的时空动态，实验证明 KRST 方法在多个现有方法上具有优势。