In this study, we present aLLM4TS, an innovative framework that adapts Large
Language Models (LLMs) for time-series representation learning. Central to our
approach is that we reconceive time-series forecasting as a self-supervised,
multi-patch prediction task, which, compared to traditional
mask-and-reconstruction methods, captures temporal dynamics in patch
representations more effectively. Our strategy encompasses two-stage training:
(i). a causal continual pre-training phase on various time-series datasets,
anchored on next patch prediction, effectively syncing LLM capabilities with
the intricacies of time-series data; (ii). fine-tuning for multi-patch
prediction in the targeted time-series context. A distinctive element of our
framework is the patch-wise decoding layer, which departs from previous methods
reliant on sequence-level decoding. Such a design directly transposes
individual patches into temporal sequences, thereby significantly bolstering
the model's proficiency in mastering temporal patch-based representations.
aLLM4TS demonstrates superior performance in several downstream tasks, proving
its effectiveness in deriving temporal representations with enhanced
transferability and marking a pivotal advancement in the adaptation of LLMs for
time-series analysis.

本研究提出了一种创新的框架 aLLM4TS，用于适应大型语言模型（LLMs）进行时间序列表示学习，并通过将时间序列预测重新构想为自监督的多补丁预测任务，比传统的遮罩和重建方法更有效地捕捉了补丁表示中的时间动态。我们的策略包括两个阶段的训练：（i）在各种时间序列数据集上基于下一个补丁预测进行因果连续预训练阶段，有效地将 LLM 的能力与时间序列数据的复杂性同步；（ii）在目标时间序列环境中进行多补丁预测的微调。我们框架的一个独特要素是基于补丁的解码层，摒弃了先前依赖序列级解码的方法。这种设计直接将各个补丁转换为时间序列，从而显著增强了模型在掌握基于时间补丁的表示方面的能力。aLLM4TS 在多个下游任务中表现出卓越性能，证明了它在提高传递性的时间表示方面的有效性，并在 LLMs 用于时间序列分析的适应性方面取得了重要进展。

多区块预测：适应时序表示学习的 LLM 调整

Multi-Patch Prediction: Adapting LLMs for Time Series Representation  Learning

In this work, we leverage pre-trained Large Language Models (LLMs) to enhance
time-series forecasting. Mirroring the growing interest in unifying models for
Natural Language Processing and Computer Vision, we envision creating an
analogous model for long-term time-series forecasting. Due to limited
large-scale time-series data for building robust foundation models, our
approach LLM4TS focuses on leveraging the strengths of pre-trained LLMs. By
combining time-series patching with temporal encoding, we have enhanced the
capability of LLMs to handle time-series data effectively. Inspired by the
supervised fine-tuning in chatbot domains, we prioritize a two-stage
fine-tuning process: first conducting supervised fine-tuning to orient the LLM
towards time-series data, followed by task-specific downstream fine-tuning.
Furthermore, to unlock the flexibility of pre-trained LLMs without extensive
parameter adjustments, we adopt several Parameter-Efficient Fine-Tuning (PEFT)
techniques. Drawing on these innovations, LLM4TS has yielded state-of-the-art
results in long-term forecasting. Our model has also shown exceptional
capabilities as both a robust representation learner and an effective few-shot
learner, thanks to the knowledge transferred from the pre-trained LLM.

我们利用预训练的大型语言模型 (LLMs) 提升时间序列预测的能力，通过结合时间序列拼接和时间编码，增强了 LLMs 处理时间序列数据的能力，采用两阶段的精调过程，并采用多种参数高效精调技术 (PEFT)，LLM4TS 在长期预测方面取得了最先进的结果。