We introduce NExT-QA, a rigorously designed video question answering
(VideoQA) benchmark to advance video understanding from describing to
explaining the temporal actions. Based on the dataset, we set up multi-choice
and open-ended QA tasks targeting causal action reasoning, temporal action
reasoning, and common scene comprehension. Through extensive analysis of
baselines and established VideoQA techniques, we find that top-performing
methods excel at shallow scene descriptions but are weak in causal and temporal
action reasoning. Furthermore, the models that are effective on multi-choice
QA, when adapted to open-ended QA, still struggle in generalizing the answers.
This raises doubt on the ability of these models to reason and highlights
possibilities for improvement. With detailed results for different question
types and heuristic observations for future works, we hope NExT-QA will guide
the next generation of VQA research to go beyond superficial scene description
towards a deeper understanding of videos. (The dataset and related resources
are available at this https URL)

介绍了 NExT-QA，它是一个具有严谨设计的视频问答基准，旨在将视频理解从描述转化为解释时间动作；通过分析基线和现有的视频问答技术，发现表现最好的方法擅长浅层场景描述，但在因果和时间动作推理方面较弱。通过详细结果和启发式观察，希望 NExT-QA 能引导下一代视频问答研究超越表面的场景描述，实现对视频更深入的理解。