This paper addresses the task of detecting and recognizing human-object interactions (HOI) in images and videos. We introduce the Graph Parsing Neural Network (GPNN), a framework that incorporates structural knowledge while being differentiable end-to-end. For a given scene, GPNN infers a parse graph that includes i) the HOI graph structure represented by an adjacency matrix, and ii) the node labels. Within a message passing inference framework, GPNN iteratively computes the adjacency matrices and node labels. We extensively evaluate our model on three HOI detection benchmarks on images and videos: HICO-DET, V-COCO, and CAD-120 datasets. Our approach significantly outperforms state-of-art methods, verifying that GPNN is scalable to large datasets and applies to spatial-temporal settings. The code is available at https://github.com/SiyuanQi/gpnn.

本文介绍了一种利用结构化知识实现可微分端到端检测和识别图像和视频中的人-物交互的算法——图解析神经网络（GPNN）。结果表明，在三个HOI检测基准测试上，我们的方法明显优于现有的最新方法，证明GPNN可以应用于大型数据集和时空设置，并可覆盖图解析和逐步计算相邻矩阵和节点标签。

通过图形解析神经网络学习人-物交互