Inspired by human neurological structures for action anticipation, we present
an action anticipation model that enables the prediction of plausible future
actions by forecasting both the visual and temporal future. In contrast to
current state-of-the-art methods which first learn a model to predict future
video features and then perform action anticipation using these features, the
proposed framework jointly learns to perform the two tasks, future visual and
temporal representation synthesis, and early action anticipation. The joint
learning framework ensures that the predicted future embeddings are informative
to the action anticipation task. Furthermore, through extensive experimental
evaluations we demonstrate the utility of using both visual and temporal
semantics of the scene, and illustrate how this representation synthesis could
be achieved through a recurrent Generative Adversarial Network (GAN) framework.
Our model outperforms the current state-of-the-art methods on multiple
datasets: UCF101, UCF101-24, UT-Interaction and TV Human Interaction.

本篇论文提出了一种行动预测模型，受人类神经结构的启发，在预测未来视觉和时间表现方面，能够预测合理的未来行动，并证明使用视觉和时间语义的场景表示结合递归生成对抗网络 (GAN) 框架可以实现未来表现的表示综合，该模型在多个数据集上优于当前最先进的方法。