As Transformer-based models have achieved impressive performance on various
time series tasks, Long-Term Series Forecasting (LTSF) tasks have also received
extensive attention in recent years. However, due to the inherent computational
complexity and long sequences demanding of Transformer-based methods, its
application on LTSF tasks still has two major issues that need to be further
investigated: 1) Whether the sparse attention mechanism designed by these
methods actually reduce the running time on real devices; 2) Whether these
models need extra long input sequences to guarantee their performance? The
answers given in this paper are negative. Therefore, to better copy with these
two issues, we design a lightweight Period-Attention mechanism (Periodformer),
which renovates the aggregation of long-term subseries via explicit periodicity
and short-term subseries via built-in proximity. Meanwhile, a gating mechanism
is embedded into Periodformer to regulate the influence of the attention module
on the prediction results. Furthermore, to take full advantage of GPUs for fast
hyperparameter optimization (e.g., finding the suitable input length), a
Multi-GPU Asynchronous parallel algorithm based on Bayesian Optimization (MABO)
is presented. MABO allocates a process to each GPU via a queue mechanism, and
then creates multiple trials at a time for asynchronous parallel search, which
greatly reduces the search time. Compared with the state-of-the-art methods,
the prediction error of Periodformer reduced by 13% and 26% for multivariate
and univariate forecasting, respectively. In addition, MABO reduces the average
search time by 46% while finding better hyperparameters. As a conclusion, this
paper indicates that LTSF may not need complex attention and extra long input
sequences. The source code will be open source on Github.

本文提出了一种轻量级的 Periodformer 模型，采用 Period-Attention 机制和内置的近似性，同时嵌入门机制控制注意模块对预测结果的影响，并使用基于贝叶斯优化的多 GPU 异步并行算法加快了超参数优化，结果表明该模型在多元和单元预测方面的预测误差分别减少了 13% 和 26%，同时缩短了 46% 的搜索时间。

长期序列预测是否需要复杂注意力和更长输入？

Does Long-Term Series Forecasting Need Complex Attention and Extra Long  Inputs?

Although Transformer-based methods have significantly improved
state-of-the-art results for long-term series forecasting, they are not only
computationally expensive but more importantly, are unable to capture the
global view of time series (e.g. overall trend). To address these problems, we
propose to combine Transformer with the seasonal-trend decomposition method, in
which the decomposition method captures the global profile of time series while
Transformers capture more detailed structures. To further enhance the
performance of Transformer for long-term prediction, we exploit the fact that
most time series tend to have a sparse representation in well-known basis such
as Fourier transform, and develop a frequency enhanced Transformer. Besides
being more effective, the proposed method, termed as Frequency Enhanced
Decomposed Transformer ({\bf FEDformer}), is more efficient than standard
Transformer with a linear complexity to the sequence length. Our empirical
studies with six benchmark datasets show that compared with state-of-the-art
methods, FEDformer can reduce prediction error by $14.8\%$ and $22.6\%$ for
multivariate and univariate time series, respectively. Code is publicly
available at this https URL

该研究提出了一种名为 FEDformer 的算法，采用 Transformer 和季节趋势分解方法相结合，提高长期预测的性能，同时通过基于 Fourier 变换的频率增强技术进一步提高了预测的准确性，与当前先进的方法相比，在多元和单元时间序列上均能大幅度降低预测误差。