We propose a multi-dimensional structured state space (S4) approach to speech
enhancement. To better capture the spectral dependencies across the frequency
axis, we focus on modifying the multi-dimensional S4 layer with whitening
transformation to build new small-footprint models that also achieve good
performance. We explore several S4-based deep architectures in time (T) and
time-frequency (TF) domains. The 2-D S4 layer can be considered a particular
convolutional layer with an infinite receptive field although it utilizes fewer
parameters than a conventional convolutional layer. Evaluated on the
VoiceBank-DEMAND data set, when compared with the conventional U-net model
based on convolutional layers, the proposed TF-domain S4-based model is 78.6%
smaller in size, yet it still achieves competitive results with a PESQ score of
3.15 with data augmentation. By increasing the model size, we can even reach a
PESQ score of 3.18.

本文提出了一种多维结构化状态空间（S4）方法来增强语音，将多维的 S4 层进行白化变换以更好地捕捉跨频率轴的频谱依赖性，探索了几种基于 S4 的 T 和 TF 深层结构，其中，在 TF 领域中，与基于卷积层的传统 U 型模型相比，所提出的 S4 模型尺寸缩小了 78.6％，但在数据增强的情况下仍然达到了竞争性结果，具有较好的 PESQ 分数。