Existing approaches for learning representations of time-series keep the
temporal arrangement of the time-steps intact with the presumption that the
original order is the most optimal for learning. However, non-adjacent sections
of real-world time-series may have strong dependencies. Accordingly we raise
the question: Is there an alternative arrangement for time-series which could
enable more effective representation learning? To address this, we propose a
simple plug-and-play mechanism called Segment, Shuffle, and Stitch (S3)
designed to improve time-series representation learning of existing models. S3
works by creating non-overlapping segments from the original sequence and
shuffling them in a learned manner that is the most optimal for the task at
hand. It then re-attaches the shuffled segments back together and performs a
learned weighted sum with the original input to capture both the newly shuffled
sequence along with the original sequence. S3 is modular and can be stacked to
create various degrees of granularity, and can be added to many forms of neural
architectures including CNNs or Transformers with negligible computation
overhead. Through extensive experiments on several datasets and
state-of-the-art baselines, we show that incorporating S3 results in
significant improvements for the tasks of time-series classification and
forecasting, improving performance on certain datasets by up to 68\%. We also
show that S3 makes the learning more stable with a smoother training loss curve
and loss landscape compared to the original baseline. The code is available at
this https URL .

现有的学习时间序列的方法保持时间步骤的时间顺序，假设原始顺序对学习来说是最优的。然而，现实世界的时间序列的非相邻部分可能具有强烈的依赖关系。为了解决这个问题，我们提出了一种名为 Segment, Shuffle, and Stitch（S3）的简单即插即用机制，旨在改进现有模型的时间序列表示学习。S3 通过从原始序列中创建非重叠的片段并以一种在任务中最优的方式对它们进行洗牌，然后将洗牌后的片段重新连接并执行与原始输入的学习加权求和，以捕获新的洗牌序列和原始序列。S3 是模块化的，可以堆叠以创建不同程度的粒度，并可添加到许多形式的神经架构，包括卷积神经网络（CNN）或 Transformer，计算开销可忽略。通过对多个数据集和现有基线进行大量实验证明，引入 S3 可以显著提高时间序列分类和预测任务的性能，在某些数据集上的性能提高高达 68％。我们还展示了 S3 相比于原始基线使学习更稳定，具有较平滑的训练损失曲线和损失曲面。代码可在此 https URL 中获得。