Earth system predictability is challenged by the complexity of environmental
dynamics and the multitude of variables involved. Current AI foundation models,
although advanced by leveraging large and heterogeneous data, are often
constrained by their size and data integration, limiting their effectiveness in
addressing the full range of Earth system prediction challenges. To overcome
these limitations, we introduce the Oak Ridge Base Foundation Model for Earth
System Predictability (ORBIT), an advanced vision-transformer model that scales
up to 113 billion parameters using a novel hybrid tensor-data orthogonal
parallelism technique. As the largest model of its kind, ORBIT surpasses the
current climate AI foundation model size by a thousandfold. Performance scaling
tests conducted on the Frontier supercomputer have demonstrated that ORBIT
achieves 230 to 707 PFLOPS, with scaling efficiency maintained at 78% to 96%
across 24,576 AMD GPUs. These breakthroughs establish new advances in AI-driven
climate modeling and demonstrate promise to significantly improve the Earth
system predictability.

ORBIT 是一种先进的视觉转换模型，具有 1130 亿个参数，通过创新的张量数据正交并行技术实现扩展。在 Frontier 超级计算机上进行的性能扩展测试表明，ORBIT 实现了 230 到 707 PFLOPS 的性能，并且在 24,576 个 AMD GPU 上保持了 78% 到 96% 的扩展效率。这些突破为基于 AI 的气候建模带来了新的进展，并展示了显著提高地球系统可预测性的前景。

ORBIT: 橡树岭基础地球系统可预测模型

ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability

Recently, the U.S. Department of Energy (DOE), Office of Science, Biological
and Environmental Research (BER), and Advanced Scientific Computing Research
(ASCR) programs organized and held the Artificial Intelligence for Earth System
Predictability (AI4ESP) workshop series. From this workshop, a critical
conclusion that the DOE BER and ASCR community came to is the requirement to
develop a new paradigm for Earth system predictability focused on enabling
artificial intelligence (AI) across the field, lab, modeling, and analysis
activities, called ModEx. The BER's `Model-Experimentation', ModEx, is an
iterative approach that enables process models to generate hypotheses. The
developed hypotheses inform field and laboratory efforts to collect measurement
and observation data, which are subsequently used to parameterize, drive, and
test model (e.g., process-based) predictions. A total of 17 technical sessions
were held in this AI4ESP workshop series. This paper discusses the topic of the
`AI Architectures and Co-design' session and associated outcomes. The AI
Architectures and Co-design session included two invited talks, two plenary
discussion panels, and three breakout rooms that covered specific topics,
including: (1) DOE HPC Systems, (2) Cloud HPC Systems, and (3) Edge computing
and Internet of Things (IoT). We also provide forward-looking ideas and
perspectives on potential research in this co-design area that can be achieved
by synergies with the other 16 session topics. These ideas include topics such
as: (1) reimagining co-design, (2) data acquisition to distribution, (3)
heterogeneous HPC solutions for integration of AI/ML and other data analytics
like uncertainty quantification with earth system modeling and simulation, and
(4) AI-enabled sensor integration into earth system measurements and
observations. Such perspectives are a distinguishing aspect of this paper.

该论文讨论了人工智能在地球系统可预测性方面的作用，提出了一种称为 ModEx 的新兴技术，介绍了人工智能架构和与其他 16 个会话主题协同研究的潜在研究方向，并提出了集成 AI/ML 和其他数据分析技术的异构高性能计算解决方案，以及 AI 传感器与地球系统测量和观测的融合。