Time series prediction plays a crucial role in various industrial fields. In
recent years, neural networks with a transformer backbone have achieved
remarkable success in many domains, including computer vision and NLP. In time
series analysis domain, some studies have suggested that even the simplest MLP
networks outperform advanced transformer-based networks on time series forecast
tasks. However, we believe these findings indicate there to be low-rank
properties in time series sequences. In this paper, we consider the low-pass
characteristics of transformers and try to incorporate the advantages of MLP.
We adopt skip-layer connections inspired by Unet into traditional transformer
backbone, thus preserving high-frequency context from input to output, namely
U-shaped Transformer. We introduce patch merge and split operation to extract
features with different scales and use larger datasets to fully make use of the
transformer backbone. Our experiments demonstrate that the model performs at an
advanced level across multiple datasets with relatively low cost.

时间序列预测是各个工业领域中的关键任务。近年来，具备变换器骨干的神经网络在计算机视觉和自然语言处理等多个领域中取得了显著成功。在时间序列分析领域，一些研究发现即使是最简单的多层感知机网络在时间序列预测任务上也能胜过先进的基于变换器的网络。然而，我们认为这些发现表明时间序列序列中存在低秩特性。本文中，我们考虑了变换器的低通特性，并试图融入多层感知机的优点。我们在传统的变换器骨干中采用了受 Unet 启发的跳跃连接，从而保留了从输入到输出的高频率上下文，即 U 型变换器。我们引入了补丁合并和分割操作，以提取不同尺度的特征，并利用更大的数据集充分利用变换器骨干。我们的实验表明，该模型在多个数据集上表现出较高水平的性能，并具有相对较低的成本。