We present TerraMind, the first any-to-any generative, multimodal foundation model for Earth observation (EO). Unlike other multimodal models, TerraMind is pretrained on dual-scale representations combining both token-level and pixel-level data across modalities. On a token level, TerraMind encodes high-level contextual information to learn cross-modal relationships, while on a pixel level, TerraMind leverages fine-grained representations to capture critical spatial nuances. We pretrained TerraMind on nine geospatial modalities of a global, large-scale dataset. In this paper, we demonstrate that (i) TerraMind's dual-scale early fusion approach unlocks a range of zero-shot and few-shot applications for Earth observation, (ii) TerraMind introduces "Thinking-in-Modalities" (TiM) -- the capability of generating additional artificial data during finetuning and inference to improve the model output -- and (iii) TerraMind achieves beyond state-of-the-art performance in community-standard benchmarks for EO like PANGAEA. The pretraining dataset, the model weights, and our code is open-sourced under a permissive license.

本研究解决了地球观测中缺乏强大生成能力的多模态模型的问题。TerraMind通过双尺度表示，结合了标记级和像素级的数据，提出了“思维-模态”（TiM）的新方法，能够在微调和推断过程中生成附加的人工数据。该模型在多个标准基准测试中超越了最先进的表现，具有广泛的零-shot和少-shot应用潜力。

TerraMind：大规模地球观测生成多模态模型