Over the past decade, Time Series Classification (TSC) has gained an
increasing attention. While various methods were explored, deep learning -
particularly through Convolutional Neural Networks (CNNs)-stands out as an
effective approach. However, due to the limited availability of training data,
defining a foundation model for TSC that overcomes the overfitting problem is
still a challenging task. The UCR archive, encompassing a wide spectrum of
datasets ranging from motion recognition to ECG-based heart disease detection,
serves as a prime example for exploring this issue in diverse TSC scenarios. In
this paper, we address the overfitting challenge by introducing pre-trained
domain foundation models. A key aspect of our methodology is a novel pretext
task that spans multiple datasets. This task is designed to identify the
originating dataset of each time series sample, with the goal of creating
flexible convolution filters that can be applied across different datasets. The
research process consists of two phases: a pre-training phase where the model
acquires general features through the pretext task, and a subsequent
fine-tuning phase for specific dataset classifications. Our extensive
experiments on the UCR archive demonstrate that this pre-training strategy
significantly outperforms the conventional training approach without
pre-training. This strategy effectively reduces overfitting in small datasets
and provides an efficient route for adapting these models to new datasets, thus
advancing the capabilities of deep learning in TSC.

该研究旨在解决时间序列分类中的过拟合问题，通过引入预训练的领域基础模型，并使用一种新颖的预训练任务，该任务可跨多个数据集，以产生可应用于不同数据集的灵活卷积滤波器。实验证明，该预训练策略显著优于传统的无预训练方法，并在小数据集上有效减少过拟合，为深度学习在时间序列分类中的应用提供了更高的性能。