Convolutional neural networks (CNN) have led to remarkable results in various fields. In this scheme, a signal is convolved with learned filters and a non-linear function is applied on the response map. The obtained result is then fed to another layer that operates similarly, thereby creating a multi-layered structure. Despite its empirical success, a theoretical understanding of this scheme, termed forward pass, is lacking. Another popular paradigm is the sparse representation model, which assumes that a signal can be described as the multiplication of a dictionary by a sparse vector. A special case of this is the convolutional sparse coding (CSC) model, in which the dictionary assumes a convolutional structure. Unlike CNN, sparsity inspired models are accompanied by a thorough theoretical analysis. Indeed, such a study of the CSC model has been performed in a recent two-part work, establishing it as a reliable alternative to the common patch-based processing. Herein, we leverage the study of the CSC model, and bring a fresh view to CNN with a deeper theoretical understanding. Our analysis relies on the observation that akin to the signal, the sparse vector can also be modeled as a sparse composition of yet another set of atoms from a convolutional dictionary. This can be extended to more than two layers, resulting in our proposed multi-layered convolutional sparse model. In this work we address the following questions: 1) What is the relation between the CNN and the proposed model? 2) In particular, can we interpret the forward pass as a pursuit? 3) If so, can we leverage this connection to provide a theoretical foundation for the forward pass? Specifically, is this algorithm guaranteed to succeed under certain conditions? Is it stable to slight perturbations in its input? 4) Lastly, can we leverage the answers to the above, and propose alternatives to CNN's forward pass?

提出了一种新的多层模型，ML-CSC，其信号被假定为从一系列CSC层中出现。CNN的前向传递实际上是服务于ML-CSC模型的阈值追踪，将CNN与ML-CSC模型紧密联系起来，为CNN带来了新的视角，同时也提出了一个与去卷积网络相关的前向通道替代方案，具有更好的理论保证。

通过卷积稀疏编码分析卷积神经网络