Time series analysis is vital for numerous applications, and transformers
have become increasingly prominent in this domain. Leading methods customize
the transformer architecture from NLP and CV, utilizing a patching technique to
convert continuous signals into segments. Yet, time series data are uniquely
challenging due to significant distribution shifts and intrinsic noise levels.
To address these two challenges,we introduce the Sparse Vector Quantized
FFN-Free Transformer (Sparse-VQ). Our methodology capitalizes on a sparse
vector quantization technique coupled with Reverse Instance Normalization
(RevIN) to reduce noise impact and capture sufficient statistics for
forecasting, serving as an alternative to the Feed-Forward layer (FFN) in the
transformer architecture. Our FFN-free approach trims the parameter count,
enhancing computational efficiency and reducing overfitting. Through
evaluations across ten benchmark datasets, including the newly introduced CAISO
dataset, Sparse-VQ surpasses leading models with a 7.84% and 4.17% decrease in
MAE for univariate and multivariate time series forecasting, respectively.
Moreover, it can be seamlessly integrated with existing transformer-based
models to elevate their performance.

稀疏量化后的无前馈层变压器（Sparse-VQ）用于时间序列分析与预测，成功地减少了噪声的影响并提高了计算效率，通过与现有的变压器模型进行集成，进一步提升了性能。