We show implicit filter level sparsity manifests in convolutional neural networks (CNNs) which employ Batch Normalization and ReLU activation, and are trained with adaptive gradient descent techniques and L2 regularization or weight decay. Through an extensive empirical study (Mehta et al., 2019) we hypothesize the mechanism behind the sparsification process, and find surprising links to certain filter sparsification heuristics proposed in literature. Emergence of, and the subsequent pruning of selective features is observed to be one of the contributing mechanisms, leading to feature sparsity at par or better than certain explicit sparsification / pruning approaches. In this workshop article we summarize our findings, and point out corollaries of selective-featurepenalization which could also be employed as heuristics for filter pruning

研究发现，在采用批归一化和ReLU激活以及使用自适应梯度下降技术和L2正则化或权值衰减训练的卷积神经网络中，存在着隐式过滤器级别稀疏性，这可能与某些在文献中提出的过滤器稀疏化启发式的特定联系有关。进一步观察表明，特定功能的出现和随后的修剪是导致特征稀疏化的机制之一，导致的特征稀疏性与某些显式稀疏化/修剪方法相当甚至更好。本文总结了研究发现，并指出了选择性功能处罚的推论，这也可作为过滤器修剪的启发式方法。

卷积神经网络中的隐式滤波器稀疏化