Transformers have gained popularity in time series forecasting for their
ability to capture long-sequence interactions. However, their high memory and
computing requirements pose a critical bottleneck for long-term forecasting. To
address this, we propose TSMixer, a lightweight neural architecture exclusively
composed of multi-layer perceptron (MLP) modules. TSMixer is designed for
multivariate forecasting and representation learning on patched time series,
providing an efficient alternative to Transformers. Our model draws inspiration
from the success of MLP-Mixer models in computer vision. We demonstrate the
challenges involved in adapting Vision MLP-Mixer for time series and introduce
empirically validated components to enhance accuracy. This includes a novel
design paradigm of attaching online reconciliation heads to the MLP-Mixer
backbone, for explicitly modeling the time-series properties such as hierarchy
and channel-correlations. We also propose a Hybrid channel modeling approach to
effectively handle noisy channel interactions and generalization across diverse
datasets, a common challenge in existing patch channel-mixing methods.
Additionally, a simple gated attention mechanism is introduced in the backbone
to prioritize important features. By incorporating these lightweight
components, we significantly enhance the learning capability of simple MLP
structures, outperforming complex Transformer models with minimal computing
usage. Moreover, TSMixer's modular design enables compatibility with both
supervised and masked self-supervised learning methods, making it a promising
building block for time-series Foundation Models. TSMixer outperforms
state-of-the-art MLP and Transformer models in forecasting by a considerable
margin of 8-60%. It also outperforms the latest strong benchmarks of
Patch-Transformer models (by 1-2%) with a significant reduction in memory and
runtime (2-3X).

TSMixer 是一个轻量级的神经架构，由多层感知机模块组成，旨在为补丁时间序列提供高效的替代转换器。通过添加在线对账头，混合通道建模和简单的门控关注机制等组件，TSMixer 的学习能力得到显著提高。TSMixer 在预测方面的性能优于现有的 MLP 和 Transformer 模型，同时可显著减少存储器和运行时间。

TSMixer：多元时间序列预测的轻量级 MLP-Mixer 模型

TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series  Forecasting

Real-world time-series datasets are often multivariate with complex dynamics.
Commonly-used high capacity architectures like recurrent- or attention-based
sequential models have become popular. However, recent work demonstrates that
simple univariate linear models can outperform those deep alternatives. In this
paper, we investigate the capabilities of linear models for time-series
forecasting and present Time-Series Mixer (TSMixer), an architecture designed
by stacking multi-layer perceptrons (MLPs). TSMixer is based on mixing
operations along time and feature dimensions to extract information
efficiently. On popular academic benchmarks, the simple-to-implement TSMixer is
comparable to specialized state-of-the-art models that leverage the inductive
biases of specific benchmarks. On the challenging and large scale M5 benchmark,
a real-world retail dataset, TSMixer demonstrates superior performance compared
to the state-of-the-art alternatives. Our results underline the importance of
efficiently utilizing cross-variate and auxiliary information for improving the
performance of time series forecasting. The design paradigms utilized in
TSMixer are expected to open new horizons for deep learning-based time series
forecasting.

本文研究线性模型在时间序列预测中的能力，提出了一个基于多层感知机的模型 TSMixer，通过在时间和特征维度上的混合操作提取信息。TSMixer 在常见的学术基准测试中表现出和专业模型类似的性能，在真实世界的 M5 基准测试数据集上表现出比先进模型更好的性能。本文的结果强调了有效利用交叉变量和辅助信息以提高时间序列预测性能的重要性，并且预计 TSMixer 的设计范例会为基于深度学习的时间序列预测开启新的视野。