Time series forecasting is an important problem and plays a key role in a
variety of applications including weather forecasting, stock market, and
scientific simulations. Although transformers have proven to be effective in
capturing dependency, its quadratic complexity of attention mechanism prevents
its further adoption in long-range time series forecasting, thus limiting them
attend to short-range range. Recent progress on state space models (SSMs) have
shown impressive performance on modeling long range dependency due to their
subquadratic complexity. Mamba, as a representative SSM, enjoys linear time
complexity and has achieved strong scalability on tasks that requires scaling
to long sequences, such as language, audio, and genomics. In this paper, we
propose to leverage a hybrid framework Mambaformer that internally combines
Mamba for long-range dependency, and Transformer for short range dependency,
for long-short range forecasting. To the best of our knowledge, this is the
first paper to combine Mamba and Transformer architecture in time series data.
We investigate possible hybrid architectures to combine Mamba layer and
attention layer for long-short range time series forecasting. The comparative
study shows that the Mambaformer family can outperform Mamba and Transformer in
long-short range time series forecasting problem. The code is available at
this https URL

在这篇论文中，我们提出了一种混合框架 Mambaformer，该框架在内部结合了 Mamba 和 Transformer 架构，用于长短范围的时间序列预测，并通过比较研究表明，Mambaformer 系列可以在长短范围的时间序列预测问题中胜过 Mamba 和 Transformer。