Temporal relational modeling in video is essential for human action
understanding, such as action recognition and action segmentation. Although
Graph Convolution Networks (GCNs) have shown promising advantages in relation
reasoning on many tasks, it is still a challenge to apply graph convolution
networks on long video sequences effectively. The main reason is that large
number of nodes (i.e., video frames) makes GCNs hard to capture and model
temporal relations in videos. To tackle this problem, in this paper, we
introduce an effective GCN module, Dilated Temporal Graph Reasoning Module
(DTGRM), designed to model temporal relations and dependencies between video
frames at various time spans. In particular, we capture and model temporal
relations via constructing multi-level dilated temporal graphs where the nodes
represent frames from different moments in video. Moreover, to enhance temporal
reasoning ability of the proposed model, an auxiliary self-supervised task is
proposed to encourage the dilated temporal graph reasoning module to find and
correct wrong temporal relations in videos. Our DTGRM model outperforms
state-of-the-art action segmentation models on three challenging datasets:
50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.
The code is available at this https URL

本篇论文提出了一种有效的 GCN 模块 DTGRM，用于捕获和建模视频帧之间的各个时间跨度的时间关系，并通过提出的自监督任务来增强模型的时间推理能力，这种模块在行为分割方面表现优于现有技术。