This paper focuses on multi-person action forecasting in videos. More
precisely, given a history of H previous frames, the goal is to detect actors
and to predict their future actions for the next T frames. Our approach jointly
models temporal and spatial interactions among different actors by constructing
a recurrent graph, using actor proposals obtained with Faster R-CNN as nodes.
Our method learns to select a subset of discriminative relations without
requiring explicit supervision, thus enabling us to tackle challenging visual
data. We refer to our model as Discriminative Relational Recurrent Network
(DRRN). Evaluation of action prediction on AVA demonstrates the effectiveness
of our proposed method compared to simpler baselines. Furthermore, we
significantly improve performance on the task of early action classification on
J-HMDB, from the previous SOTA of 48% to 60%.

本文旨在研究视频中多人行为预测。文章提出了一种 Discriminative Relational Recurrent Network 模型，通过构建循环图形式对不同演员之间的时空相互作用进行联合建模，从而在不需要显式监督的情况下，学习选择相关性。该方法在 AVA 数据集上预测动作的评估和 J-HMDB 任务的性能都有显著提高。