Sparse observations and coarse-resolution climate models limit effective regional decision-making, underscoring the need for robust downscaling. However, existing AI methods struggle with generalization across variables and geographies and are constrained by the quadratic complexity of Vision Transformer (ViT) self-attention. We introduce ORBIT-2, a scalable foundation model for global, hyper-resolution climate downscaling. ORBIT-2 incorporates two key innovations: (1) Residual Slim ViT (Reslim), a lightweight architecture with residual learning and Bayesian regularization for efficient, robust prediction; and (2) TILES, a tile-wise sequence scaling algorithm that reduces self-attention complexity from quadratic to linear, enabling long-sequence processing and massive parallelism. ORBIT-2 scales to 10 billion parameters across 32,768 GPUs, achieving up to 1.8 ExaFLOPS sustained throughput and 92-98% strong scaling efficiency. It supports downscaling to 0.9 km global resolution and processes sequences up to 4.2 billion tokens. On 7 km resolution benchmarks, ORBIT-2 achieves high accuracy with R^2 scores in the range of 0.98 to 0.99 against observation data.

本文解决了稀疏观测和粗分辨率气候模型在区域决策中的局限性，通过提出ORBIT-2，一个可扩展的全球超高分辨率气候下尺度化基础模型。该模型的创新点在于引入Residual Slim ViT和TILES算法，使自注意力复杂度从平方降至线性，显著提升了处理效率与准确性。实验表明，ORBIT-2在7公里分辨率的基准测试中，R^2评分达到了0.98至0.99，展示了其在气候预测中的潜在影响力。

ORBIT-2：可扩展的超大规模视觉基础模型用于天气和气候下尺度化