As Transformer-based models have achieved impressive performance on various
time series tasks, Long-Term Series Forecasting (LTSF) tasks have also received
extensive attention in recent years. However, due to the inherent computational
complexity and long sequences demanding of Transformer-based methods, its
application on LTSF tasks still has two major issues that need to be further
investigated: 1) Whether the sparse attention mechanism designed by these
methods actually reduce the running time on real devices; 2) Whether these
models need extra long input sequences to guarantee their performance? The
answers given in this paper are negative. Therefore, to better copy with these
two issues, we design a lightweight Period-Attention mechanism (Periodformer),
which renovates the aggregation of long-term subseries via explicit periodicity
and short-term subseries via built-in proximity. Meanwhile, a gating mechanism
is embedded into Periodformer to regulate the influence of the attention module
on the prediction results. Furthermore, to take full advantage of GPUs for fast
hyperparameter optimization (e.g., finding the suitable input length), a
Multi-GPU Asynchronous parallel algorithm based on Bayesian Optimization (MABO)
is presented. MABO allocates a process to each GPU via a queue mechanism, and
then creates multiple trials at a time for asynchronous parallel search, which
greatly reduces the search time. Compared with the state-of-the-art methods,
the prediction error of Periodformer reduced by 13% and 26% for multivariate
and univariate forecasting, respectively. In addition, MABO reduces the average
search time by 46% while finding better hyperparameters. As a conclusion, this
paper indicates that LTSF may not need complex attention and extra long input
sequences. The source code will be open source on Github.

本文提出了一种轻量级的 Periodformer 模型，采用 Period-Attention 机制和内置的近似性，同时嵌入门机制控制注意模块对预测结果的影响，并使用基于贝叶斯优化的多 GPU 异步并行算法加快了超参数优化，结果表明该模型在多元和单元预测方面的预测误差分别减少了 13% 和 26%，同时缩短了 46% 的搜索时间。