In recent years, there has been a growing interest in integrating linear
state-space models (SSM) in deep neural network architectures of foundation
models. This is exemplified by the recent success of Mamba, showing better
performance than the state-of-the-art Transformer architectures in language
tasks. Foundation models, like e.g. GPT-4, aim to encode sequential data into a
latent space in order to learn a compressed representation of the data. The
same goal has been pursued by control theorists using SSMs to efficiently model
dynamical systems. Therefore, SSMs can be naturally connected to deep sequence
modeling, offering the opportunity to create synergies between the
corresponding research areas. This paper is intended as a gentle introduction
to SSM-based architectures for control theorists and summarizes the latest
research developments. It provides a systematic review of the most successful
SSM proposals and highlights their main features from a control theoretic
perspective. Additionally, we present a comparative analysis of these models,
evaluating their performance on a standardized benchmark designed for assessing
a model's efficiency at learning long sequences.

近年来，将线性状态空间模型（SSM）整合到深度神经网络架构的基础模型中的兴趣日益增长。该论文对控制理论家介绍了 SSM 基于架构，并总结了最新的研究进展，对最成功的 SSM 提案进行了系统回顾，从控制理论的角度突出了它们的主要特点。此外，我们针对用于评估模型在学习长序列方面效率的标准基准进行了这些模型的性能比较分析。

状态空间模型作为基础模型的控制论概述

State Space Models as Foundation Models: A Control Theoretic Overview

Banded matrices can be used as precision matrices in several models including
linear state-space models, some Gaussian processes, and Gaussian Markov random
fields. The aim of the paper is to make modern inference methods (such as
variational inference or gradient-based sampling) available for Gaussian models
with banded precision. We show that this can efficiently be achieved by
equipping an automatic differentiation framework, such as TensorFlow or
PyTorch, with some linear algebra operators dedicated to banded matrices. This
paper studies the algorithmic aspects of the required operators, details their
reverse-mode derivatives, and show that their complexity is linear in the
number of observations.

本文的主要研究领域是将带状矩阵用于高斯模型的精度矩阵中，旨在通过配备专门用于带状矩阵的线性代数算子来实现现代推理方法的运用，同时研究这些算子的算法方面，详细阐述它们的反向传播导数，并证明它们的复杂度与观测数量呈线性关系。