Video action anticipation aims to predict future action categories from
observed frames. Current state-of-the-art approaches mainly resort to recurrent
neural networks to encode history information into hidden states, and predict
future actions from the hidden representations. It is well known that the
recurrent pipeline is inefficient in capturing long-term information which may
limit its performance in predication task. To address this problem, this paper
proposes a simple yet efficient Temporal Transformer with Progressive
Prediction (TTPP) framework, which repurposes a Transformer-style architecture
to aggregate observed features, and then leverages a light-weight network to
progressively predict future features and actions. Specifically, predicted
features along with predicted probabilities are accumulated into the inputs of
subsequent prediction. We evaluate our approach on three action datasets,
namely TVSeries, THUMOS-14, and TV-Human-Interaction. Additionally we also
conduct a comprehensive study for several popular aggregation and prediction
strategies. Extensive results show that TTPP not only outperforms the
state-of-the-art methods but also more efficient.

本文提出一个名为 TTPP 的框架，使用 Transformer 风格的架构，通过汇总观察到的特征，并利用轻量的网络逐步预测未来的特征和行动，来解决当前流行的基于循环神经网络的视频行动预测模型中无法有效捕获长期信息的缺点，在三个数据集上的广泛结果表明，TTPP 不仅胜过现有技术方法，而且更有效。