The field of Earth Observations (EO) offers a wealth of data from diverse
sensors, presenting a great opportunity for advancing self-supervised
multimodal learning. However, current multimodal EO datasets and models focus
on a single data type, either mono-date images or time series, which limits
their expressivity. We introduce OmniSat, a novel architecture that exploits
the spatial alignment between multiple EO modalities to learn expressive
multimodal representations without labels. To demonstrate the advantages of
combining modalities of different natures, we augment two existing datasets
with new modalities. As demonstrated on three downstream tasks: forestry, land
cover classification, and crop mapping. OmniSat can learn rich representations
in an unsupervised manner, leading to improved performance in the semi- and
fully-supervised settings, even when only one modality is available for
inference. The code and dataset are available at github.com/gastruc/OmniSat.

利用地球观测数据的多模态性质，提出了一种无监督多模态学习方法 OmniSat，可用于改进森林学、土地覆盖分类和农作物映射等任务，并在半监督和全监督设置下获得更好的性能。