We propose a multi-dimensional structured state space (S4) approach to speech
enhancement. To better capture the spectral dependencies across the frequency
axis, we focus on modifying the multi-dimensional S4 layer with whitening
transformation to build new small-footprint models that also achieve good
performance. We explore several S4-based deep architectures in time (T) and
time-frequency (TF) domains. The 2-D S4 layer can be considered a particular
convolutional layer with an infinite receptive field although it utilizes fewer
parameters than a conventional convolutional layer. Evaluated on the
VoiceBank-DEMAND data set, when compared with the conventional U-net model
based on convolutional layers, the proposed TF-domain S4-based model is 78.6%
smaller in size, yet it still achieves competitive results with a PESQ score of
3.15 with data augmentation. By increasing the model size, we can even reach a
PESQ score of 3.18.

本文提出了一种多维结构化状态空间（S4）方法来增强语音，将多维的 S4 层进行白化变换以更好地捕捉跨频率轴的频谱依赖性，探索了几种基于 S4 的 T 和 TF 深层结构，其中，在 TF 领域中，与基于卷积层的传统 U 型模型相比，所提出的 S4 模型尺寸缩小了 78.6％，但在数据增强的情况下仍然达到了竞争性结果，具有较好的 PESQ 分数。

利用小型模型的多维深层结构状态空间方法进行语音增强

A Multi-dimensional Deep Structured State Space Approach to Speech  Enhancement Using Small-footprint Models

Models using structured state space sequence (S4) layers have achieved
state-of-the-art performance on long-range sequence modeling tasks. An S4 layer
combines linear state space models (SSMs), the HiPPO framework, and deep
learning to achieve high performance. We build on the design of the S4 layer
and introduce a new state space layer, the S5 layer. Whereas an S4 layer uses
many independent single-input, single-output SSMs, the S5 layer uses one
multi-input, multi-output SSM. We establish a connection between S5 and S4, and
use this to develop the initialization and parameterization used by the S5
model. The result is a state space layer that can leverage efficient and widely
implemented parallel scans, allowing S5 to match the computational efficiency
of S4, while also achieving state-of-the-art performance on several long-range
sequence modeling tasks. S5 averages 87.4% on the long range arena benchmark,
and 98.5% on the most difficult Path-X task.

介绍了基于深度学习、HiPPO 形式和线性状态空间模型的结构化状态空间序列（S4）层，以及在其基础上引入了一种新的状态空间层 ——S5 层。与 S4 层使用多个独立的 SSM 相比，S5 层使用一个多输入、多输出的 SSM，从而实现了更高的计算效率和性能。S5 在长距离序列建模任务上达到了最新的最高性能，并使最难的 Path-X 任务的准确率达到了 98.5%。