Long-term traffic prediction has always been a challenging task due to its
dynamic temporal dependencies and complex spatial dependencies. In this paper,
we propose a model that combines hybrid Transformer and spatio-temporal
self-supervised learning. The model enhances its robustness by applying
adaptive data augmentation techniques at the sequence-level and graph-level of
the traffic data. It utilizes Transformer to overcome the limitations of
recurrent neural networks in capturing long-term sequences, and employs
Chebyshev polynomial graph convolution to capture complex spatial dependencies.
Furthermore, considering the impact of spatio-temporal heterogeneity on traffic
speed, we design two self-supervised learning tasks to model the temporal and
spatial heterogeneity, thereby improving the accuracy and generalization
ability of the model. Experimental evaluations are conducted on two real-world
datasets, PeMS04 and PeMS08, and the results are visualized and analyzed,
demonstrating the superior performance of the proposed model.

利用混合 Transformer 和时空自监督学习的模型来提高长期交通预测的鲁棒性，该模型通过在交通数据的序列级和图级应用自适应数据增强技术来增强其鲁棒性，利用 Transformer 克服了循环神经网络在捕捉长期序列方面的局限性，并采用 Chebyshev 多项式图卷积来捕捉复杂的空间依赖关系。此外，考虑到时空异质性对交通速度的影响，我们设计了两个自监督学习任务来建模时空异质性，从而提高模型的准确性和泛化能力。在两个真实数据集 PeMS04 和 PeMS08 上进行了实验评估，结果进行了可视化和分析，证明了所提模型的卓越性能。

融合 Transformer 和时空自监督学习的长期交通预测

Hybrid Transformer and Spatial-Temporal Self-Supervised Learning for  Long-term Traffic Prediction

Accurate forecasting of multivariate traffic flow time series remains
challenging due to substantial spatio-temporal heterogeneity and complex
long-range correlative patterns. To address this, we propose
Spatio-Temporal-Decoupled Masked Pre-training (STD-MAE), a novel framework that
employs masked autoencoders to learn and encode complex spatio-temporal
dependencies via pre-training. Specifically, we use two decoupled masked
autoencoders to reconstruct the traffic data along spatial and temporal axes
using a self-supervised pre-training approach. These mask reconstruction
mechanisms capture the long-range correlations in space and time separately.
The learned hidden representations are then used to augment the downstream
spatio-temporal traffic predictor. A series of quantitative and qualitative
evaluations on four widely-used traffic benchmarks (PEMS03, PEMS04, PEMS07, and
PEMS08) are conducted to verify the state-of-the-art performance, with STD-MAE
explicitly enhancing the downstream spatio-temporal models' ability to capture
long-range intricate spatial and temporal patterns. Codes are available at
this https URL

通过使用掩码自编码器进行预训练，我们提出了一种新颖的框架 STD-MAE，用于准确预测多变量交通流时间序列，该框架能够学习和编码复杂的时空依赖关系，并通过隐含表示增强下游时空交通预测模型的能力。