Large pre-trained models have been instrumental in significant advancements
in domains like language and vision making model training for individual
downstream tasks more efficient as well as provide superior performance.
However, tackling time-series analysis tasks usually involves designing and
training a separate model from scratch leveraging training data and domain
expertise specific to the task. We tackle a significant challenge for
pre-training a general time-series model from multiple heterogeneous
time-series dataset: providing semantically useful inputs to models for
modeling time series of different dynamics from different domains. We observe
that partitioning time-series into segments as inputs to sequential models
produces semantically better inputs and propose a novel model LPTM that
automatically identifies optimal dataset-specific segmentation strategy
leveraging self-supervised learning loss during pre-training. LPTM provides
performance similar to or better than domain-specific state-of-art model and is
significantly more data and compute efficient taking up to 40% less data as
well as 50% less training time to achieve state-of-art performance in a wide
range of time-series analysis tasks from multiple disparate domain.

将多个不同动态和领域的时间序列分割为片段作为输入，我们提出了一种新颖的模型 LPTM 来进行预训练，自动识别最佳的数据集特定分段策略，并在多个不同领域的时间序列分析任务中实现达到甚至超过领域特定最先进模型的性能，并且数据和计算效率明显提高，使用的数据量减少了 40％，训练时间减少了 50％。