In this work, we explore the intersection of sparse coding theory and deep learning to enhance our understanding of feature extraction capabilities in advanced neural network architectures. We begin by introducing a novel class of Deep Sparse Coding (DSC) models and establish a thorough theoretical analysis of their uniqueness and stability properties. By applying iterative algorithms to these DSC models, we derive convergence rates for convolutional neural networks (CNNs) in their ability to extract sparse features. This provides a strong theoretical foundation for the use of CNNs in sparse feature learning tasks. We additionally extend this convergence analysis to more general neural network architectures, including those with diverse activation functions, as well as self-attention and transformer-based models. This broadens the applicability of our findings to a wide range of deep learning methods for deep sparse feature extraction. Inspired by the strong connection between sparse coding and CNNs, we also explore training strategies to encourage neural networks to learn more sparse features. Through numerical experiments, we demonstrate the effectiveness of these approaches, providing valuable insights for the design of efficient and interpretable deep learning models.

本研究探讨了稀疏编码理论与深度学习的交集，旨在提升对高级神经网络架构中特征提取能力的理解。我们提出了一类新型深度稀疏编码模型，并建立了其唯一性和稳定性性质的彻底理论分析，揭示了其在稀疏特征学习任务中的收敛速率，从而为卷积神经网络在稀疏特征提取中的应用奠定了坚实的理论基础。

基于卷积神经网络的深度稀疏编码收敛性分析