In recent years, there has been a rapid development of spatio-temporal
prediction techniques in response to the increasing demands of traffic
management and travel planning. While advanced end-to-end models have achieved
notable success in improving predictive performance, their integration and
expansion pose significant challenges. This work aims to address these
challenges by introducing a spatio-temporal pre-training framework that
seamlessly integrates with downstream baselines and enhances their performance.
The framework is built upon two key designs: (i) We propose a spatio-temporal
mask autoencoder as a pre-training model for learning spatio-temporal
dependencies. The model incorporates customized parameter learners and
hierarchical spatial pattern encoding networks. These modules are specifically
designed to capture spatio-temporal customized representations and intra- and
inter-cluster region semantic relationships, which have often been neglected in
existing approaches. (ii) We introduce an adaptive mask strategy as part of the
pre-training mechanism. This strategy guides the mask autoencoder in learning
robust spatio-temporal representations and facilitates the modeling of
different relationships, ranging from intra-cluster to inter-cluster, in an
easy-to-hard training manner. Extensive experiments conducted on representative
benchmarks demonstrate the effectiveness of our proposed method. We have made
our model implementation publicly available at this https URL

该研究提出了一个融入下游基线模型并提升其性能的时空预训练框架，其中包括一个时空蒙版自编码器和一个自适应蒙版策略，通过学习时空依赖关系和不同关系的建模，在交通管理和旅行规划方面取得了显著的效果。