Masked time series modeling has recently gained much attention as a
self-supervised representation learning strategy for time series. Inspired by
masked image modeling in computer vision, recent works first patchify and
partially mask out time series, and then train Transformers to capture the
dependencies between patches by predicting masked patches from unmasked
patches. However, we argue that capturing such patch dependencies might not be
an optimal strategy for time series representation learning; rather, learning
to embed patches independently results in better time series representations.
Specifically, we propose to use 1) the simple patch reconstruction task, which
autoencode each patch without looking at other patches, and 2) the simple
patch-wise MLP that embeds each patch independently. In addition, we introduce
complementary contrastive learning to hierarchically capture adjacent time
series information efficiently. Our proposed method improves time series
forecasting and classification performance compared to state-of-the-art
Transformer-based models, while it is more efficient in terms of the number of
parameters and training/inference time. Code is available at this repository:
this https URL

基于自监督表征学习策略的遮蔽时间序列建模引起了人们的广泛关注。我们提出使用简单的补丁重构任务和独立嵌入每个补丁的简单补丁级 MLP。此外，我们引入互补的对比学习方法以高效地捕捉相邻时间序列信息。与最先进的基于 Transformer 的模型相比，我们的方法在时间序列预测和分类性能上有所提高，同时在参数数量和训练 / 推理时间上更加高效。