In the domain of video question answering (VideoQA), the impact of question types on VQA systems, despite its critical importance, has been relatively under-explored to date. However, the richness of question types directly determines the range of concepts a model needs to learn, thereby affecting the upper limit of its learning capability. This paper focuses on exploring the significance of different question types for VQA systems and their impact on performance, revealing a series of issues such as insufficient learning and model degradation due to uneven distribution of question types. Particularly, considering the significant variation in dependency on temporal information across different question types, and given that the representation of such information coincidentally represents a principal challenge and difficulty for VideoQA as opposed to ImageQA. To address these challenges, we propose QTG-VQA, a novel architecture that incorporates question-type-guided attention and adaptive learning mechanism. Specifically, as to temporal-type questions, we design Masking Frame Modeling technique to enhance temporal modeling, aimed at encouraging the model to grasp richer visual-language relationships and manage more intricate temporal dependencies. Furthermore, a novel evaluation metric tailored to question types is introduced. Experimental results confirm the effectiveness of our approach.

本研究针对视频问答系统中不同问题类型对性能的影响进行探讨，凸显了现有研究中对问题类型探索不足的问题。提出QTG-VQA架构，通过问题类型引导的注意机制和自适应学习手段，特别设计了针对时间型问题的掩蔽帧建模技术，以增强模型对时间信息的理解和高效学习。实验验证了该方法的有效性，为VideoQA领域提供了新的评估标准和改进方向。

QTG-VQA：问题类型引导的视频问答系统架构