Prior studies on Remote Sensing Foundation Model (RSFM) reveal immense
potential towards a generic model for Earth Observation. Nevertheless, these
works primarily focus on a single modality without temporal and geo-context
modeling, hampering their capabilities for diverse tasks. In this study, we
present SkySense, a generic billion-scale model, pre-trained on a curated
multi-modal Remote Sensing Imagery (RSI) dataset with 21.5 million temporal
sequences. SkySense incorporates a factorized multi-modal spatiotemporal
encoder taking temporal sequences of optical and Synthetic Aperture Radar (SAR)
data as input. This encoder is pre-trained by our proposed Multi-Granularity
Contrastive Learning to learn representations across different modal and
spatial granularities. To further enhance the RSI representations by the
geo-context clue, we introduce Geo-Context Prototype Learning to learn
region-aware prototypes upon RSI's multi-modal spatiotemporal features. To our
best knowledge, SkySense is the largest Multi-Modal RSFM to date, whose modules
can be flexibly combined or used individually to accommodate various tasks. It
demonstrates remarkable generalization capabilities on a thorough evaluation
encompassing 16 datasets over 7 tasks, from single- to multi-modal, static to
temporal, and classification to localization. SkySense surpasses 18 recent
RSFMs in all test scenarios. Specifically, it outperforms the latest models
such as GFM, SatLas and Scale-MAE by a large margin, i.e., 2.76%, 3.67% and
3.61% on average respectively. We will release the pre-trained weights to
facilitate future research and Earth Observation applications.

SkySense 是目前最大的多模态遥感基础模型，通过多种模态的时空序列输入，采用 Multi-Granularity Contrastive Learning 和 Geo-Context Prototype Learning 进行预训练学习，能灵活结合或单独使用其模块以适应各种任务，并在 16 个数据集上的 7 个任务中表现出了卓越的泛化能力，超过了 18 个最新的遥感基础模型。