Weather forecasting is a long-standing computational challenge with direct
societal and economic impacts. This task involves a large amount of continuous
data collection and exhibits rich spatiotemporal dependencies over long
periods, making it highly suitable for deep learning models. In this paper, we
apply pre-training techniques to weather forecasting and propose W-MAE, a
Weather model with Masked AutoEncoder pre-training for multi-variable weather
forecasting. W-MAE is pre-trained in a self-supervised manner to reconstruct
spatial correlations within meteorological variables. On the temporal scale, we
fine-tune the pre-trained W-MAE to predict the future states of meteorological
variables, thereby modeling the temporal dependencies present in weather data.
We pre-train W-MAE using the fifth-generation ECMWF Reanalysis (ERA5) data,
with samples selected every six hours and using only two years of data. Under
the same training data conditions, we compare W-MAE with FourCastNet, and W-MAE
outperforms FourCastNet in precipitation forecasting. In the setting where the
training data is far less than that of FourCastNet, our model still performs
much better in precipitation prediction (0.80 vs. 0.98). Additionally,
experiments show that our model has a stable and significant advantage in
short-to-medium-range forecasting (i.e., forecasting time ranges from 6 hours
to one week), and the longer the prediction time, the more evident the
performance advantage of W-MAE, further proving its robustness.

通过预训练技术，我们提出了一种基于自编码器的气象模型 W-MAE 用于多变量天气预报，在长时间范围内建模了气象变量的空间相关性，在时间尺度上，我们利用预训练的 W-MAE 对气象变量进行未来状态的预测，与 FourCastNet 模型进行比较，W-MAE 具有更好的降雨预测性能，特别是在短到中期范围内的预测中。