Large Pretrained models for Zero/Few-shot learning excel in language and
vision domains but encounter challenges in multivariate time series (TS) due to
the diverse nature and scarcity of publicly available pretraining data.
Consequently, there has been a recent surge in utilizing pretrained large
language models (LLMs) with various adaptations for time series forecasting.
These approaches employ cross-domain transfer learning, yielding highly
impressive results. However, these models are typically very large ($\sim$
billion parameters), exhibit slow execution, and do not consider cross-channel
correlations. To address this, we present Multi-level Tiny Time Mixers (TTM), a
significantly smaller model based on the lightweight TSMixer architecture. TTM
marks the first success in developing tiny pretrained models ($\le$1 million
parameters), exclusively trained on public TS data with effective transfer
learning capabilities. To tackle the complexity of pretraining on multiple
datasets with varied temporal resolutions, we introduce several novel
enhancements such as adaptive patching, dataset augmentation via downsampling,
and resolution prefix tuning. Moreover, we employ a multi-level modeling
strategy to effectively model channel correlations and incorporate exogenous
signals during finetuning, a crucial capability lacking in existing benchmarks.
TTM excels in few/zero-shot forecasting, demonstrating significant accuracy
gains (12-38%) over existing benchmarks. Further, it achieves a remarkable
14-106X reduction in model parameters, enabling 54-65X faster
training/inference as compared to the LLM-TS benchmarks. In fact, TTM's
zero-shot results often surpass the few-shot results in many benchmarks,
highlighting the efficacy of our approach. Code and Pretrained Models will be
open-sourced.

利用预训练模型进行时间序列预测方面的研究，提出了 Multi-level Tiny Time Mixers（TTM）方法，通过小型模型、自适应补丁、数据增强和分辨率前缀调整来有效地模拟跨通道相关性和整合外部信号，实现了显著的准确性提升和参数规模大幅减小。