Self-supervised representation learning of Multivariate Time Series (MTS) is
a challenging task and attracts increasing research interests in recent years.
Many previous works focus on the pretext task of self-supervised learning and
usually neglect the complex problem of MTS encoding, leading to unpromising
results. In this paper, we tackle this challenge from two aspects: encoder and
pretext task, and propose a unified channel-aware self-supervised learning
framework CaSS. Specifically, we first design a new Transformer-based encoder
Channel-aware Transformer (CaT) to capture the complex relationships between
different time channels of MTS. Second, we combine two novel pretext tasks Next
Trend Prediction (NTP) and Contextual Similarity (CS) for the self-supervised
representation learning with our proposed encoder. Extensive experiments are
conducted on several commonly used benchmark datasets. The experimental results
show that our framework achieves new state-of-the-art comparing with previous
self-supervised MTS representation learning methods (up to +7.70\% improvement
on LSST dataset) and can be well applied to the downstream MTS classification.

本文提出了一种统一的通道感知自监督学习框架（CaSS），通过设计一个新的基于 Transformer 的编码器（CaT）来捕捉 MTS 不同时间通道之间的复杂关系，并结合 Next Trend Prediction（NTP）和 Contextual Similarity（CS）两种新颖的预训练任务来实现自监督学习的表示学习，实验结果表明，与以前的自监督 MTS 表示学习方法相比，我们的框架在几个常用的基准数据集上均达到了新的最优成果（LSST 数据集上提高了 7.70％），并可很好地应用于下游 MTS 分类。