Self-supervised learning for time-series data holds potential similar to that
recently unleashed in Natural Language Processing and Computer Vision. While
most existing works in this area focus on contrastive learning, we propose a
conceptually simple yet powerful non-contrastive approach, based on the
data2vec self-distillation framework. The core of our method is a
student-teacher scheme that predicts the latent representation of an input time
series from masked views of the same time series. This strategy avoids strong
modality-specific assumptions and biases typically introduced by the design of
contrastive sample pairs. We demonstrate the competitiveness of our approach
for classification and forecasting as downstream tasks, comparing with
state-of-the-art self-supervised learning methods on the UCR and UEA archives
as well as the ETT and Electricity datasets.

时间序列上的自我监督学习能够使得类似于自然语言处理和计算机视觉领域中最近释放的潜力发挥出来。通过使用基于 data2vec 自蒸馏框架的一个概念上简单而强大的非对比度方法，我们的方法展示了在分类和预测等下游任务上与 UCR、UEA 和 ETT 以及 Electricity 数据集上的最先进的自我监督学习方法相比的竞争力。