Time series forecasting is crucial for applications across multiple domains
and various scenarios. Although Transformer models have dramatically shifted
the landscape of forecasting, their effectiveness remains debated. Recent
findings have indicated that simpler linear models might outperform complex
Transformer-based approaches, highlighting the potential for more streamlined
architectures. In this paper, we shift focus from the overall architecture of
the Transformer to the effectiveness of self-attentions for time series
forecasting. To this end, we introduce a new architecture, Cross-Attention-only
Time Series transformer (CATS), that rethinks the traditional Transformer
framework by eliminating self-attention and leveraging cross-attention
mechanisms instead. By establishing future horizon-dependent parameters as
queries and enhanced parameter sharing, our model not only improves long-term
forecasting accuracy but also reduces the number of parameters and memory
usage. Extensive experiment across various datasets demonstrates that our model
achieves superior performance with the lowest mean squared error and uses fewer
parameters compared to existing models.

我们介绍了一个新的架构，交叉注意力时间序列变换器（CATS），它通过消除自注意力并利用交叉注意力机制来重新思考传统 Transformer 框架，不仅提高长期预测精度，还减少参数和内存使用。大量对各种数据集进行的实验表明，我们的模型在最小均方误差和参数使用方面实现了卓越的性能。