In this work we pretrain a CLIP/ViT based model using three different
modalities of satellite imagery across five AOIs covering over ~10\% of the
earth total landmass, namely Sentinel 2 RGB optical imagery, Sentinel 1 SAR
amplitude and Sentinel 1 SAR interferometric coherence. This model uses $\sim
250$ M parameters. Then, we use the embeddings produced for each modality with
a classical machine learning method to attempt different downstream tasks for
earth observation related to vegetation, built up surface, croplands and
permanent water. We consistently show how we reduce the need for labeled data
by 99\%, so that with ~200-500 randomly selected labeled examples (around
4K-10K km$^2$) we reach performance levels analogous to those achieved with the
full labeled datasets (about 150K image chips or 3M km$^2$ in each AOI) on all
modalities, AOIs and downstream tasks. This leads us to think that the model
has captured significant earth features useful in a wide variety of scenarios.
To enhance our model's usability in practice, its architecture allows inference
in contexts with missing modalities and even missing channels within each
modality. Additionally, we visually show that this embedding space, obtained
with no labels, is sensible to the different earth features represented by the
labelled datasets we selected.

使用 CLIP/ViT 模型在占总陆地面积约 10% 的五个区域内，利用卫星图像的三种不同模态对其进行预训练，通过嵌入向量和经典机器学习方法，在植被、建筑表面、农田和常水的地球观测相关任务中，仅需少量标记数据即可达到与完整标记数据相当的性能水平，并且该模型对缺失的数据模态和通道具备容错性。