Visual interactivity understanding within visual scenes presents a significant challenge in computer vision. Existing methods focus on complex interactivities while leveraging a simple relationship model. These methods, however, struggle with a diversity of appearance, situation, position, interaction, and relation in videos. This limitation hinders the ability to fully comprehend the interplay within the complex visual dynamics of subjects. In this paper, we delve into interactivities understanding within visual content by deriving scene graph representations from dense interactivities among humans and objects. To achieve this goal, we first present a new dataset containing Appearance-Situation-Position-Interaction-Relation predicates, named ASPIRe, offering an extensive collection of videos marked by a wide range of interactivities. Then, we propose a new approach named Hierarchical Interlacement Graph (HIG), which leverages a unified layer and graph within a hierarchical structure to provide deep insights into scene changes across five distinct tasks. Our approach demonstrates superior performance to other methods through extensive experiments conducted in various scenarios.

在本文中，我们从人类和物体之间的密集互动中推导出场景图表示，以探索视觉内容中的互动理解。为了实现这一目标，我们首先提出了一个含有广泛互动标记视频的新数据集，称为ASPIRe，并提出了一种名为Hierarchical Interlacement Graph（HIG）的新方法，通过分层结构中的统一层和图，深入研究场景变化的五个不同任务。通过在各种场景中进行的广泛实验，我们的方法表现出优越的性能。

HIG：视频理解中的层次交织图场景图生成方法