Spatiotemporal predictive learning offers a self-supervised learning paradigm
that enables models to learn both spatial and temporal patterns by predicting
future sequences based on historical sequences. Mainstream methods are
dominated by recurrent units, yet they are limited by their lack of
parallelization and often underperform in real-world scenarios. To improve
prediction quality while maintaining computational efficiency, we propose an
innovative triplet attention transformer designed to capture both inter-frame
dynamics and intra-frame static features. Specifically, the model incorporates
the Triplet Attention Module (TAM), which replaces traditional recurrent units
by exploring self-attention mechanisms in temporal, spatial, and channel
dimensions. In this configuration: (i) temporal tokens contain abstract
representations of inter-frame, facilitating the capture of inherent temporal
dependencies; (ii) spatial and channel attention combine to refine the
intra-frame representation by performing fine-grained interactions across
spatial and channel dimensions. Alternating temporal, spatial, and
channel-level attention allows our approach to learn more complex short- and
long-range spatiotemporal dependencies. Extensive experiments demonstrate
performance surpassing existing recurrent-based and recurrent-free methods,
achieving state-of-the-art under multi-scenario examination including moving
object trajectory prediction, traffic flow prediction, driving scene
prediction, and human motion capture.

基于历史序列预测未来序列的时空预测学习提供了一种自监督学习范式，主流方法利用循环单元进行建模，但循环单元的并行性不足，常常在现实场景中表现欠佳。为了在保持计算效率的同时提高预测质量，我们提出了一种创新的三元注意力变换器，在设计上捕捉了帧间动态与帧内静态特征。通过将 Triplet Attention Module (TAM) 整合到模型中，我们取代了传统的循环单元，并对时空和通道维度中的自注意力机制进行了深入探索。在这种配置下：(i) 时序标记包含了帧间的抽象表示，有助于捕捉固有的时序依赖性；(ii) 空间和通道的注意力结合，通过在空间和通道维度上进行细粒度交互来改进帧内表示。交替运用时序、空间和通道级别的注意力使得我们的方法能够学习更复杂的短程和长程时空依赖关系。广泛的实验表明，我们的方法在移动物体轨迹预测、交通流预测、驾驶场景预测和人体动作捕捉等多种场景下性能超过了现有的基于循环和非循环方法，达到了最先进水平。