We introduce a novel learning method for Structured State Space Sequence (S4)
models incorporating Diagonal State Space (DSS) layers, tailored for processing
long-sequence data in edge intelligence applications, including sensor data
analysis and real-time analytics. This method utilizes the balanced truncation
technique, prevalent in control theory, applied specifically to DSS layers to
reduce computational costs during inference. By leveraging parameters from the
reduced model, we refine the initialization process of S4 models, outperforming
the widely used Skew-HiPPo initialization in terms of performance. Numerical
experiments demonstrate that our trained S4 models with DSS layers surpass
conventionally trained models in accuracy and efficiency metrics. Furthermore,
our observations reveal a positive correlation: higher accuracy in the original
model consistently leads to increased accuracy in models trained using our
method, suggesting that our approach effectively leverages the strengths of the
original model.

我们引入了一种新颖的学习方法，用于处理边缘智能应用中的长序列数据，包括传感器数据分析和实时分析，并在结构化状态空间序列模型中引入了对角状态空间层。该方法利用控制理论中常见的平衡截断技术，特别应用于对角状态空间层，以减少推理过程中的计算开销。通过利用减少模型的参数，我们改进了 S4 模型的初始化过程，在性能方面优于广泛使用的 Skew-HiPPo 初始化方法。数值实验表明，使用对角状态空间层训练的 S4 模型在准确度和效率指标上超过了传统训练的模型。此外，我们的观察结果显示了一个正向相关性：原始模型的准确度越高，使用我们的方法训练的模型的准确度也越高，这表明我们的方法有效地利用了原始模型的优势。