One of the guiding principles for designing AI-based weather forecasting
systems is to embed physical constraints as inductive priors in the neural
network architecture. A popular prior is locality, where the atmospheric data
is processed with local neural interactions, like 3D convolutions or 3D local
attention windows as in Pangu-Weather. On the other hand, some works have shown
great success in weather forecasting without this locality principle, at the
cost of a much higher parameter count.
In this paper, we show that the 3D local processing in Pangu-Weather is
computationally sub-optimal. We design ArchesWeather, a transformer model that
combines 2D attention with a column-wise attention-based feature interaction
module, and demonstrate that this design improves forecasting skill.
ArchesWeather is trained at 1.5{\deg} resolution and 24h lead time, with a
training budget of a few GPU-days and a lower inference cost than competing
methods. An ensemble of two of our best models shows competitive RMSE scores
with the IFS HRES and outperforms the 1.4{\deg} 50-members NeuralGCM ensemble
for one day ahead forecasting.
Code and models will be made publicly available at
this https URL

本研究提出了一种基于 AI 的天气预测系统，通过将物理约束作为归纳先验嵌入到神经网络架构中，将 2D 注意力与基于列的注意力特征交互模块相结合，设计出一种名为 ArchesWeather 的转换器模型，证明了该设计改进了预测技巧。