We propose an efficient design of Transformer-based models for multivariate
time series forecasting and self-supervised representation learning. It is
based on two key components: (i) segmentation of time series into
subseries-level patches which are served as input tokens to Transformer; (ii)
channel-independence where each channel contains a single univariate time
series that shares the same embedding and Transformer weights across all the
series. Patching design naturally has three-fold benefit: local semantic
information is retained in the embedding; computation and memory usage of the
attention maps are quadratically reduced given the same look-back window; and
the model can attend longer history. Our channel-independent patch time series
Transformer (PatchTST) can improve the long-term forecasting accuracy
significantly when compared with that of SOTA Transformer-based models. We also
apply our model to self-supervised pre-training tasks and attain excellent
fine-tuning performance, which outperforms supervised training on large
datasets. Transferring of masked pre-trained representation on one dataset to
others also produces SOTA forecasting accuracy. Code is available at:
this https URL.

本文提出了一种基于 Transformer 的多元时间序列预测和自我监督表征学习的有效设计，它基于两个关键组件：时间序列划分为子序列级别的片段，这些片段作为输入标记传递给 Transformer；通道独立性，其中每个通道包含一个单变量时间序列，其在所有系列中共享相同的嵌入和 Transformer 权重。PatchTST 可以显著提高长期预测准确性，超越 SOTA Transformer-based models，并在自我监督预训练任务中实现出色的微调性能。