Time Series Representation Learning (TSRL) focuses on generating informative
representations for various Time Series (TS) modeling tasks. Traditional
Self-Supervised Learning (SSL) methods in TSRL fall into four main categories:
reconstructive, adversarial, contrastive, and predictive, each with a common
challenge of sensitivity to noise and intricate data nuances. Recently,
diffusion-based methods have shown advanced generative capabilities. However,
they primarily target specific application scenarios like imputation and
forecasting, leaving a gap in leveraging diffusion models for generic TSRL. Our
work, Time Series Diffusion Embedding (TSDE), bridges this gap as the first
diffusion-based SSL TSRL approach. TSDE segments TS data into observed and
masked parts using an Imputation-Interpolation-Forecasting (IIF) mask. It
applies a trainable embedding function, featuring dual-orthogonal Transformer
encoders with a crossover mechanism, to the observed part. We train a reverse
diffusion process conditioned on the embeddings, designed to predict noise
added to the masked part. Extensive experiments demonstrate TSDE's superiority
in imputation, interpolation, forecasting, anomaly detection, classification,
and clustering. We also conduct an ablation study, present embedding
visualizations, and compare inference speed, further substantiating TSDE's
efficiency and validity in learning representations of TS data.

通过自我监督学习方法构建的时间序列扩散嵌入模型 (Time Series Diffusion Embedding, TSDE) 在时间序列表示学习领域具有出色的性能，在缺失数据插补、插值、预测、异常检测、分类和聚类等任务上表现优异。

通过扩散过程和填补 - 插值 - 预测掩码进行时间序列表示的自监督学习

Self-Supervised Learning of Time Series Representation via Diffusion  Process and Imputation-Interpolation-Forecasting Mask

Modern techniques like contrastive learning have been effectively used in
many areas, including computer vision, natural language processing, and
graph-structured data. Creating positive examples that assist the model in
learning robust and discriminative representations is a crucial stage in
contrastive learning approaches. Usually, preset human intuition directs the
selection of relevant data augmentations. Due to patterns that are easily
recognized by humans, this rule of thumb works well in the vision and language
domains. However, it is impractical to visually inspect the temporal structures
in time series. The diversity of time series augmentations at both the dataset
and instance levels makes it difficult to choose meaningful augmentations on
the fly. In this study, we address this gap by analyzing time series data
augmentation using information theory and summarizing the most commonly adopted
augmentations in a unified format. We then propose a contrastive learning
framework with parametric augmentation, AutoTCL, which can be adaptively
employed to support time series representation learning. The proposed approach
is encoder-agnostic, allowing it to be seamlessly integrated with different
backbone encoders. Experiments on univariate forecasting tasks demonstrate the
highly competitive results of our method, with an average 6.5\% reduction in
MSE and 4.7\% in MAE over the leading baselines. In classification tasks,
AutoTCL achieves a $1.2\%$ increase in average accuracy.

通过分析时间序列数据增强使用信息理论，并总结最常采用的增强方法，我们提出了一种参数增强的对比学习框架 AutoTCL，它可以自适应地支持时间序列表示学习，无缝集成在不同的主干编码器中，实验证明在一元预测任务和分类任务中，我们的方法分别比领先的基准方法平均降低 6.5% 和 4.7% 的误差，并提高 1.2% 的平均准确率。