Inspired by Large Language Models (LLMs), Time Series Forecasting (TSF), a
long-standing task in time series analysis, is undergoing a transition towards
Large Time Series Models (LTSMs), aiming to train universal transformer-based
models for TSF. However, training LTSMs on heterogeneous time series data poses
unique challenges, including diverse frequencies, dimensions, and patterns
across datasets. Recent endeavors have studied and evaluated various design
choices aimed at enhancing LTSM training and generalization capabilities,
spanning pre-processing techniques, model configurations, and dataset
configurations. In this work, we comprehensively analyze these design choices
and aim to identify the best practices for training LTSM. Moreover, we propose
\emph{time series prompt}, a novel statistical prompting strategy tailored to
time series data. Furthermore, based on the observations in our analysis, we
introduce \texttt{LTSM-bundle}, which bundles the best design choices we have
identified. Empirical results demonstrate that \texttt{LTSM-bundle} achieves
superior zero-shot and few-shot performances compared to state-of-the-art LSTMs
and traditional TSF methods on benchmark datasets.

基于大型语言模型（LLMs）的启发，时间序列预测（TSF）正在向基于 Universal-Transformer 的大型时间序列模型（LTSMs）过渡，旨在为 TSF 训练通用模型。然而，在异构时间序列数据上训练 LTSMs 面临着多样的挑战，包括数据集之间不同的频率，维度和模式。我们通过综合分析设计选择，并旨在确定训练 LTSM 的最佳实践。此外，我们提出了一种针对时间序列数据的新型统计提示策略称为 “时间序列提示”。基于我们分析中的观察结果，我们引入了 “LTSM-bundle”，该捆绑包绑定了我们确定的最佳设计选择。实证结果表明，“LTSM-bundle” 在基准数据集上相比于最先进的 LSTMs 和传统的 TSF 方法具有卓越的零射击和少射击性能。

大型时间序列模型训练中不同设计选择的理解

Understanding Different Design Choices in Training Large Time Series  Models

Machine learning models have made significant progress in load forecasting,
but their forecast accuracy is limited in cases where historical load data is
scarce. Inspired by the outstanding performance of large language models (LLMs)
in computer vision and natural language processing, this paper aims to discuss
the potential of large time series models in load forecasting with scarce
historical data. Specifically, the large time series model is constructed as a
time series generative pre-trained transformer (TimeGPT), which is trained on
massive and diverse time series datasets consisting of 100 billion data points
(e.g., finance, transportation, banking, web traffic, weather, energy,
healthcare, etc.). Then, the scarce historical load data is used to fine-tune
the TimeGPT, which helps it to adapt to the data distribution and
characteristics associated with load forecasting. Simulation results show that
TimeGPT outperforms the benchmarks (e.g., popular machine learning models and
statistical models) for load forecasting on several real datasets with scarce
training samples, particularly for short look-ahead times. However, it cannot
be guaranteed that TimeGPT is always superior to benchmarks for load
forecasting with scarce data, since the performance of TimeGPT may be affected
by the distribution differences between the load data and the training data. In
practical applications, we can divide the historical data into a training set
and a validation set, and then use the validation set loss to decide whether
TimeGPT is the best choice for a specific dataset.

研究表明在历史电负荷数据有限的情况下，机器学习模型在负荷预测中的准确性受到限制。该论文探讨了大规模时间序列模型（TimeGPT）在历史数据有限的负荷预测中的潜力，并通过训练模型适应数据分布和特征，取得了在短期预测中超越其他机器学习模型和统计模型的成果。然而，由于负荷数据与训练数据之间的分布差异可能会影响 TimeGPT 的性能，因此在实际应用中需要使用验证集损失来确定是否选择 TimeGPT。