Time Series Representation Learning (TSRL) focuses on generating informative
representations for various Time Series (TS) modeling tasks. Traditional
Self-Supervised Learning (SSL) methods in TSRL fall into four main categories:
reconstructive, adversarial, contrastive, and predictive, each with a common
challenge of sensitivity to noise and intricate data nuances. Recently,
diffusion-based methods have shown advanced generative capabilities. However,
they primarily target specific application scenarios like imputation and
forecasting, leaving a gap in leveraging diffusion models for generic TSRL. Our
work, Time Series Diffusion Embedding (TSDE), bridges this gap as the first
diffusion-based SSL TSRL approach. TSDE segments TS data into observed and
masked parts using an Imputation-Interpolation-Forecasting (IIF) mask. It
applies a trainable embedding function, featuring dual-orthogonal Transformer
encoders with a crossover mechanism, to the observed part. We train a reverse
diffusion process conditioned on the embeddings, designed to predict noise
added to the masked part. Extensive experiments demonstrate TSDE's superiority
in imputation, interpolation, forecasting, anomaly detection, classification,
and clustering. We also conduct an ablation study, present embedding
visualizations, and compare inference speed, further substantiating TSDE's
efficiency and validity in learning representations of TS data.

通过自我监督学习方法构建的时间序列扩散嵌入模型 (Time Series Diffusion Embedding, TSDE) 在时间序列表示学习领域具有出色的性能，在缺失数据插补、插值、预测、异常检测、分类和聚类等任务上表现优异。