The Transformer model has shown leading performance in time series
forecasting. Nevertheless, in some complex scenarios, it tends to learn
low-frequency features in the data and overlook high-frequency features,
showing a frequency bias. This bias prevents the model from accurately
capturing important high-frequency data features. In this paper, we undertook
empirical analyses to understand this bias and discovered that frequency bias
results from the model disproportionately focusing on frequency features with
higher energy. Based on our analysis, we formulate this bias and propose
Fredformer, a Transformer-based framework designed to mitigate frequency bias
by learning features equally across different frequency bands. This approach
prevents the model from overlooking lower amplitude features important for
accurate forecasting. Extensive experiments show the effectiveness of our
proposed approach, which can outperform other baselines in different real-world
time-series datasets. Furthermore, we introduce a lightweight variant of the
Fredformer with an attention matrix approximation, which achieves comparable
performance but with much fewer parameters and lower computation costs. The
code is available at: this https URL

本文介绍了一种基于 Transformer 的框架 Fredformer，用于准确预测时间序列数据，该框架通过均匀地学习不同频带的特征来减轻频率偏差，并且引入了一个轻量级变种的 Fredformer，其参数更少、计算成本更低，但性能仍然可与其他基线方法相比。