Long-term time-series forecasting (LTTF) has become a pressing demand in many
applications, such as wind power supply planning. Transformer models have been
adopted to deliver high prediction capacity because of the high computational
self-attention mechanism. Though one could lower the complexity of Transformers
by inducing the sparsity in point-wise self-attentions for LTTF, the limited
information utilization prohibits the model from exploring the complex
dependencies comprehensively. To this end, we propose an efficient
Transformerbased model, named Conformer, which differentiates itself from
existing methods for LTTF in three aspects: (i) an encoder-decoder architecture
incorporating a linear complexity without sacrificing information utilization
is proposed on top of sliding-window attention and Stationary and Instant
Recurrent Network (SIRN); (ii) a module derived from the normalizing flow is
devised to further improve the information utilization by inferring the outputs
with the latent variables in SIRN directly; (iii) the inter-series correlation
and temporal dynamics in time-series data are modeled explicitly to fuel the
downstream self-attention mechanism. Extensive experiments on seven real-world
datasets demonstrate that Conformer outperforms the state-of-the-art methods on
LTTF and generates reliable prediction results with uncertainty quantification.

本论文讨论了长期时间序列预测 (LTTF) 中的 Transformer 模型，提出了一种有效的基于 Transformer 的模型 Conformer，采用编码器 - 解码器架构和正则化流派生的模块进一步提高信息利用率，并显式地建模时间序列数据中的互系列相关性和时间动态以加强下游自我关注机制，实验表明 Conformer 模型在 LTTF 中优于现有的方法，并可生成可靠的预测结果与不确定性量化。

实现长期时间序列预测：特征、模式和分布

Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution

Continuous-depth neural networks, such as the Neural Ordinary Differential
Equations (ODEs), have aroused a great deal of interest from the communities of
machine learning and data science in recent years, which bridge the connection
between deep neural networks and dynamical systems. In this article, we
introduce a new sort of continuous-depth neural network, called the Neural
Piecewise-Constant Delay Differential Equations (PCDDEs). Here, unlike the
recently proposed framework of the Neural Delay Differential Equations (DDEs),
we transform the single delay into the piecewise-constant delay(s). The Neural
PCDDEs with such a transformation, on one hand, inherit the strength of
universal approximating capability in Neural DDEs. On the other hand, the
Neural PCDDEs, leveraging the contributions of the information from the
multiple previous time steps, further promote the modeling capability without
augmenting the network dimension. With such a promotion, we show that the
Neural PCDDEs do outperform the several existing continuous-depth neural
frameworks on the one-dimensional piecewise-constant delay population dynamics
and real-world datasets, including MNIST, CIFAR10, and SVHN.

介绍了一种新型的连续深度神经网络，名为 “神经分段常数时滞微分方程”，其中利用了信息多点的贡献，提高了模型的能力，并在 MNIST、CIFAR10 和 SVHN 等真实数据集上优于多种现有的连续深度神经网络框架。