The rapid rise of deep learning (DL) in numerical weather prediction (NWP)
has led to a proliferation of models which forecast atmospheric variables with
comparable or superior skill than traditional physics-based NWP. However, among
these leading DL models, there is a wide variance in both the training settings
and architecture used. Further, the lack of thorough ablation studies makes it
hard to discern which components are most critical to success. In this work, we
show that it is possible to attain high forecast skill even with relatively
off-the-shelf architectures, simple training procedures, and moderate compute
budgets. Specifically, we train a minimally modified SwinV2 transformer on ERA5
data, and find that it attains superior forecast skill when compared against
IFS. We present some ablations on key aspects of the training pipeline,
exploring different loss functions, model sizes and depths, and multi-step
fine-tuning to investigate their effect. We also examine the model performance
with metrics beyond the typical ACC and RMSE, and investigate how the
performance scales with model size.

通过在 ERA5 数据上训练经过最小改进的 SwinV2 Transformer 模型，我们展示了即使在相对常规的体系结构、简单的训练过程和适度的计算预算下，也可以实现高水平的预测技巧，并比较了其与 IFS 的优越性。我们对训练流程的关键方面进行了一些剖析，探索了不同的损失函数、模型尺寸和深度以及多步细调的影响，并且通过超出典型 ACC 和 RMSE 的指标来检查模型的性能，并研究了模型规模对性能的影响。

大规模基于 Transformer 的天气预测的训练配方分析与探索

Analyzing and Exploring Training Recipes for Large-Scale  Transformer-Based Weather Prediction

Analog forecasting is a nonparametric technique introduced by Lorenz in 1969
which predicts the evolution of states of a dynamical system (or observables
defined on the states) by following the evolution of the sample in a historical
record of observations which most closely resembles the current initial data.
Here, we introduce a suite of forecasting methods which improve traditional
analog forecasting by combining ideas from kernel methods developed in harmonic
analysis and machine learning and state-space reconstruction for dynamical
systems. A key ingredient of our approach is to replace single-analog
forecasting with weighted ensembles of analogs constructed using local
similarity kernels. The kernels used here employ a number of dynamics-dependent
features designed to improve forecast skill, including Takens' delay-coordinate
maps (to recover information in the initial data lost through partial
observations) and a directional dependence on the dynamical vector field
generating the data. Mathematically, our approach is closely related to kernel
methods for out-of-sample extension of functions, and we discuss alternative
strategies based on the Nystr\"om method and the multiscale Laplacian pyramids
technique. We illustrate these techniques in applications to forecasting in a
low-order deterministic model for atmospheric dynamics with chaotic
metastability, and interannual-scale forecasting in the North Pacific sector of
a comprehensive climate model. We find that forecasts based on kernel-weighted
ensembles have significantly higher skill than the conventional approach
following a single analog.

本研究提出了一套基于核方法，状态空间重建和加权模拟集的预测方法，其主要特点是使用局部相似性核构建加权集合，通过此方法大幅提高了预测准确性。