Time-series forecasting plays an important role in many real-world scenarios,
such as equipment life cycle forecasting, weather forecasting, and traffic flow
forecasting. It can be observed from recent research that a variety of
transformer-based models have shown remarkable results in time-series
forecasting. However, there are still some issues that limit the ability of
transformer-based models on time-series forecasting tasks: (i) learning
directly on raw data is susceptible to noise due to its complex and unstable
feature representation; (ii) the self-attention mechanisms pay insufficient
attention to changing features and temporal dependencies. In order to solve
these two problems, we propose a transformer-based differentially reconstructed
attention model DRAformer. Specifically, DRAformer has the following
innovations: (i) learning against differenced sequences, which preserves clear
and stable sequence features by differencing and highlights the changing
properties of sequences; (ii) the reconstructed attention: integrated distance
attention exhibits sequential distance through a learnable Gaussian kernel,
distributed difference attention calculates distribution difference by mapping
the difference sequence to the adaptive feature space, and the combination of
the two effectively focuses on the sequences with prominent associations; (iii)
the reconstructed decoder input, which extracts sequence features by
integrating variation information and temporal correlations, thereby obtaining
a more comprehensive sequence representation. Extensive experiments on four
large-scale datasets demonstrate that DRAformer outperforms state-of-the-art
baselines.

提出了一种基于转换器的差分重构注意模型 DRAformer，该模型通过使用差分进行稳定特征表示和集中关注序列属性解决时间序列预测中存在的问题，在四个大规模数据集上表现出了优越性。