In this study, we present aLLM4TS, an innovative framework that adapts Large
Language Models (LLMs) for time-series representation learning. Central to our
approach is that we reconceive time-series forecasting as a self-supervised,
multi-patch prediction task, which, compared to traditional
mask-and-reconstruction methods, captures temporal dynamics in patch
representations more effectively. Our strategy encompasses two-stage training:
(i). a causal continual pre-training phase on various time-series datasets,
anchored on next patch prediction, effectively syncing LLM capabilities with
the intricacies of time-series data; (ii). fine-tuning for multi-patch
prediction in the targeted time-series context. A distinctive element of our
framework is the patch-wise decoding layer, which departs from previous methods
reliant on sequence-level decoding. Such a design directly transposes
individual patches into temporal sequences, thereby significantly bolstering
the model's proficiency in mastering temporal patch-based representations.
aLLM4TS demonstrates superior performance in several downstream tasks, proving
its effectiveness in deriving temporal representations with enhanced
transferability and marking a pivotal advancement in the adaptation of LLMs for
time-series analysis.

本研究提出了一种创新的框架 aLLM4TS，用于适应大型语言模型（LLMs）进行时间序列表示学习，并通过将时间序列预测重新构想为自监督的多补丁预测任务，比传统的遮罩和重建方法更有效地捕捉了补丁表示中的时间动态。我们的策略包括两个阶段的训练：（i）在各种时间序列数据集上基于下一个补丁预测进行因果连续预训练阶段，有效地将 LLM 的能力与时间序列数据的复杂性同步；（ii）在目标时间序列环境中进行多补丁预测的微调。我们框架的一个独特要素是基于补丁的解码层，摒弃了先前依赖序列级解码的方法。这种设计直接将各个补丁转换为时间序列，从而显著增强了模型在掌握基于时间补丁的表示方面的能力。aLLM4TS 在多个下游任务中表现出卓越性能，证明了它在提高传递性的时间表示方面的有效性，并在 LLMs 用于时间序列分析的适应性方面取得了重要进展。