Detecting human-object interactions (HOI) is an important step toward a
comprehensive visual understanding of machines. While detecting non-temporal
HOIs (e.g., sitting on a chair) from static images is feasible, it is unlikely
even for humans to guess temporal-related HOIs (e.g., opening/closing a door)
from a single video frame, where the neighboring frames play an essential role.
However, conventional HOI methods operating on only static images have been
used to predict temporal-related interactions, which is essentially guessing
without temporal contexts and may lead to sub-optimal performance. In this
paper, we bridge this gap by detecting video-based HOIs with explicit temporal
information. We first show that a naive temporal-aware variant of a common
action detection baseline does not work on video-based HOIs due to a
feature-inconsistency issue. We then propose a simple yet effective
architecture named Spatial-Temporal HOI Detection (ST-HOI) utilizing temporal
information such as human and object trajectories, correctly-localized visual
features, and spatial-temporal masking pose features. We construct a new video
HOI benchmark dubbed VidHOI where our proposed approach serves as a solid
baseline.

本研究提出了一种简单而有效的体系结构，名为空间 - 时间 HOI 检测（ST-HOI），利用人类和对象轨迹、正确定位的视觉特征和空间 - 时间遮蔽姿态特征等时间信息来检测基于视频的人与对象之间的交互，构建了一个新的视频交互基准，被称为 VidHOI 基准。