Accurate weather forecasting is crucial in various sectors, impacting
decision-making processes and societal events. Data-driven approaches based on
machine learning models have recently emerged as a promising alternative to
numerical weather prediction models given their potential to capture physics of
different scales from historical data and the significantly lower computational
cost during the prediction stage. Renowned for its state-of-the-art performance
across diverse domains, the Transformer model has also gained popularity in
machine learning weather prediction. Yet applying Transformer architectures to
weather forecasting, particularly on a global scale is computationally
challenging due to the quadratic complexity of attention and the quadratic
increase in spatial points as resolution increases. In this work, we propose a
factorized-attention-based model tailored for spherical geometries to mitigate
this issue. More specifically, it utilizes multi-dimensional factorized kernels
that convolve over different axes where the computational complexity of the
kernel is only quadratic to the axial resolution instead of overall resolution.
The deterministic forecasting accuracy of the proposed model on $1.5^\circ$ and
0-7 days' lead time is on par with state-of-the-art purely data-driven machine
learning weather prediction models. We also showcase the proposed model holds
great potential to push forward the Pareto front of accuracy-efficiency for
Transformer weather models, where it can achieve better accuracy with less
computational cost compared to Transformer based models with standard
attention.

我们提出了一种适用于球面几何的因子化注意力模型，用于天气预测。该模型在精确度和效率上的表现与纯数据驱动的机器学习天气预测模型相当，具备在准确度与计算成本上优于基于 Transformer 模型的标准注意力模型的潜力。