Although the Transformer has been the dominant architecture for time series
forecasting tasks in recent years, a fundamental challenge remains: the
permutation-invariant self-attention mechanism within Transformers leads to a
loss of temporal information. To tackle these challenges, we propose
PatchMixer, a novel CNN-based model. It introduces a permutation-variant
convolutional structure to preserve temporal information. Diverging from
conventional CNNs in this field, which often employ multiple scales or numerous
branches, our method relies exclusively on depthwise separable convolutions.
This allows us to extract both local features and global correlations using a
single-scale architecture. Furthermore, we employ dual forecasting heads that
encompass both linear and nonlinear components to better model future curve
trends and details. Our experimental results on seven time-series forecasting
benchmarks indicate that compared with the state-of-the-art method and the
best-performing CNN, PatchMixer yields $3.9\%$ and $21.2\%$ relative
improvements, respectively, while being 2-3x faster than the most advanced
method. We will release our code and model.

通过引入一种新颖的 CNN 模型 ——PatchMixer，我们解决了 Transformer 模型在时间序列预测任务中面临的挑战，该模型具有可以保留时间信息的排列不变自注意力机制。与传统 CNN 不同的是，我们的方法仅依赖于深度可分离卷积，可以在单一尺度的结构中提取局部特征和全局相关性。试验结果表明，与现有最先进的方法和表现最佳的 CNN 相比，PatchMixer 相对提升了分别为 3.9％和 21.2％，而且速度是最先进方法的 2-3 倍。我们将发布我们的代码和模型。