It has been well recognized that modeling human-object or object-object
relations would be helpful for detection task. Nevertheless, the problem is not
trivial especially when exploring the interactions between human actor, object
and scene (collectively as human-context) to boost video action detectors. The
difficulty originates from the aspect that reliable relations in a video should
depend on not only short-term human-context relation in the present clip but
also the temporal dynamics distilled over a long-range span of the video. This
motivates us to capture both short-term and long-term relations in a video. In
this paper, we present a new Long Short-Term Relation Networks, dubbed as LSTR,
that novelly aggregates and propagates relation to augment features for video
action detection. Technically, Region Proposal Networks (RPN) is remoulded to
first produce 3D bounding boxes, i.e., tubelets, in each video clip. LSTR then
models short-term human-context interactions within each clip through
spatio-temporal attention mechanism and reasons long-term temporal dynamics
across video clips via Graph Convolutional Networks (GCN) in a cascaded manner.
Extensive experiments are conducted on four benchmark datasets, and superior
results are reported when comparing to state-of-the-art methods.

本文提出了一种新的长短期关系网络（LSTR），通过空时注意机制建模视频内的短期人类 - 背景关系，并通过级联方式使用图卷积网络（GCN）推理长期时空动态，从而增强视频动作检测的特征表现。在四个基准数据集上进行广泛实验，与现有方法相比，取得了优异的结果。