Most existing forecasting systems are memory-based methods, which attempt to
mimic human forecasting ability by employing various memory mechanisms and have
progressed in temporal modeling for memory dependency. Nevertheless, an obvious
weakness of this paradigm is that it can only model limited historical
dependence and can not transcend the past. In this paper, we rethink the
temporal dependence of event evolution and propose a novel
memory-anticipation-based paradigm to model an entire temporal structure,
including the past, present, and future. Based on this idea, we present
Memory-and-Anticipation Transformer (MAT), a memory-anticipation-based
approach, to address the online action detection and anticipation tasks. In
addition, owing to the inherent superiority of MAT, it can process online
action detection and anticipation tasks in a unified manner. The proposed MAT
model is tested on four challenging benchmarks TVSeries, THUMOS'14, HDD, and
EPIC-Kitchens-100, for online action detection and anticipation tasks, and it
significantly outperforms all existing methods. Code is available at
this https URL

该论文提出了一种基于记忆和预测的方法来建模整个时间结构，包括过去、现在和未来，并通过 Memory-and-Anticipation Transformer 在在线动作检测和预测任务中取得显著的性能优势。