Operational weather forecasting system relies on computationally expensive physics-based models. Although Transformers-based models have shown remarkable potential in weather forecasting, Transformers are discrete models which limit their ability to learn the continuous spatio-temporal features of the dynamical weather system. We address this issue with Conformer, a spatio-temporal Continuous Vision Transformer for weather forecasting. Conformer is designed to learn the continuous weather evolution over time by implementing continuity in the multi-head attention mechanism. The attention mechanism is encoded as a differentiable function in the transformer architecture to model the complex weather dynamics. We evaluate Conformer against a state-of-the-art Numerical Weather Prediction (NWP) model and several deep learning based weather forecasting models. Conformer outperforms some of the existing data-driven models at all lead times while only being trained at lower resolution data.

以Transformer为基础的模型在天气预报中显示出了显著的潜力，但受到离散模型的限制，其学习连续的时空特征方面能力有限。本文提出了一种用于天气预报的时空连续视觉Transformer模型Conformer，通过在多头注意力机制中实现连续性来学习连续的天气演变过程。在与一种先进的数值天气预报模型和其他深度学习基于天气预报模型的对比中，Conformer在所有的时间引导下表现出优于部分现有数据驱动模型的性能，且仅使用分辨率较低的数据进行训练。

Conformer：将连续注意力嵌入视觉Transformer以用于天气预测