Although the Transformer has been the dominant architecture for time series
forecasting tasks in recent years, a fundamental challenge remains: the
permutation-invariant self-attention mechanism within Transformers leads to a
loss of temporal information. To tackle these challenges, we propose
PatchMixer, a novel CNN-based model. It introduces a permutation-variant
convolutional structure to preserve temporal information. Diverging from
conventional CNNs in this field, which often employ multiple scales or numerous
branches, our method relies exclusively on depthwise separable convolutions.
This allows us to extract both local features and global correlations using a
single-scale architecture. Furthermore, we employ dual forecasting heads that
encompass both linear and nonlinear components to better model future curve
trends and details. Our experimental results on seven time-series forecasting
benchmarks indicate that compared with the state-of-the-art method and the
best-performing CNN, PatchMixer yields $3.9\%$ and $21.2\%$ relative
improvements, respectively, while being 2-3x faster than the most advanced
method. We will release our code and model.

通过引入一种新颖的 CNN 模型 ——PatchMixer，我们解决了 Transformer 模型在时间序列预测任务中面临的挑战，该模型具有可以保留时间信息的排列不变自注意力机制。与传统 CNN 不同的是，我们的方法仅依赖于深度可分离卷积，可以在单一尺度的结构中提取局部特征和全局相关性。试验结果表明，与现有最先进的方法和表现最佳的 CNN 相比，PatchMixer 相对提升了分别为 3.9％和 21.2％，而且速度是最先进方法的 2-3 倍。我们将发布我们的代码和模型。

PatchMixer：长期时间序列预测的补丁混合架构

PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series  Forecasting

The recent trend in deep learning methods for 3D point cloud understanding is
to propose increasingly sophisticated architectures either to better capture 3D
geometries or by introducing possibly undesired inductive biases. Moreover,
prior works introducing novel architectures compared their performance on the
same domain, devoting less attention to their generalization to other domains.
We argue that the ability of a model to transfer the learnt knowledge to
different domains is an important feature that should be evaluated to
exhaustively assess the quality of a deep network architecture. In this work we
propose PatchMixer, a simple yet effective architecture that extends the ideas
behind the recent MLP-Mixer paper to 3D point clouds. The novelties of our
approach are the processing of local patches instead of the whole shape to
promote robustness to partial point clouds, and the aggregation of patch-wise
features using an MLP as a simpler alternative to the graph convolutions or the
attention mechanisms that are used in prior works. We evaluated our method on
the shape classification and part segmentation tasks, achieving superior
generalization performance compared to a selection of the most relevant deep
architectures.

本文提出了 PatchMixer 架构，通过处理局部补丁和使用 MLP 聚合补丁特征，将 MLP-Mixer 的思想扩展到 3D 点云中，以提高深度网络架构的泛化性能。