Multi-modality spatio-temporal (MoST) data extends spatio-temporal (ST) data
by incorporating multiple modalities, which is prevalent in monitoring systems,
encompassing diverse traffic demands and air quality assessments. Despite
significant strides in ST modeling in recent years, there remains a need to
emphasize harnessing the potential of information from different modalities.
Robust MoST forecasting is more challenging because it possesses (i)
high-dimensional and complex internal structures and (ii) dynamic heterogeneity
caused by temporal, spatial, and modality variations. In this study, we propose
a novel MoST learning framework via Self-Supervised Learning, namely MoSSL,
which aims to uncover latent patterns from temporal, spatial, and modality
perspectives while quantifying dynamic heterogeneity. Experiment results on two
real-world MoST datasets verify the superiority of our approach compared with
the state-of-the-art baselines. Model implementation is available at
this https URL

通过自监督学习提出了一种新的多模态时空学习框架，称为 MoSSl，旨在从时间、空间和模态角度揭示潜在模式，并量化动态异质性。实验结果验证了我们方法相对于现有基线的优越性。