Deep learning for time series forecasting has traditionally operated within a one-model-per-dataset framework, limiting its potential to leverage the game-changing impact of large pre-trained models. The concept of universal forecasting, emerging from pre-training on a vast collection of time series datasets, envisions a single Large Time Series Model capable of addressing diverse downstream forecasting tasks. However, constructing such a model poses unique challenges specific to time series data: i) cross-frequency learning, ii) accommodating an arbitrary number of variates for multivariate time series, and iii) addressing the varying distributional properties inherent in large-scale data. To address these challenges, we present novel enhancements to the conventional time series Transformer architecture, resulting in our proposed Masked Encoder-based Universal Time Series Forecasting Transformer (Moirai). Trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains, Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models. Code, model weights, and data will be released.

我们提出了基于掩码编码器的通用时间序列预测变形器Moirai，用于解决传统上仅限于一个数据集一个模型的深度学习时间序列预测框架的限制，利用大规模预先训练模型的革命性影响。通过在超过九个领域中提供的超过270亿次观测的大规模开放时间序列存档（LOTSA）上进行训练，Moirai作为零-shot预测器实现了具有竞争力或更好的性能。

统一训练通用时间序列预测Transformer