Human activities can be learned from video. With effective modeling it is possible to discover not only the action labels but also the temporal structures of the activities such as the progression of the sub-activities. Automatically recognizing such structure from raw video signal is a new capability that promises authentic modeling and successful recognition of human-object interactions. Toward this goal, we introduce Asynchronous-Sparse Interaction Graph Networks (ASSIGN), a recurrent graph network that is able to automatically detect the structure of interaction events associated with entities in a video scene. ASSIGN pioneers learning of autonomous behavior of video entities including their dynamic structure and their interaction with the coexisting neighbors. Entities' lives in our model are asynchronous to those of others therefore more flexible in adaptation to complex scenarios. Their interactions are sparse in time hence more faithful to the true underlying nature and more robust in inference and learning. ASSIGN is tested on human-object interaction recognition and shows superior performance in segmenting and labeling of human sub-activities and object affordances from raw videos. The native ability for discovering temporal structures of the model also eliminates the dependence on external segmentation that was previously mandatory for this task.

通过介绍一种新的循环图网络架构Asynchronous-Sparse Interaction Graph Networks，将其实施于人物与物品交互行为识别中，通过发掘视频片段中实体间的行为事件结构，自动实现视频实体与其邻居间的动态结构交互的学习，从而消除了对原处理单元的外部区分和先验知识的依赖，使其标记和分割人的子行为和物体切入更为准确，具备更强的鲁棒性。

在视频中学习异步和稀疏的人-物互动