Structured network pruning excels non-structured methods because they can
take advantage of the thriving developed parallel computing techniques. In this
paper, we propose a new structured pruning method. Firstly, to create more
structured redundancy, we present a data-driven loss function term calculated
from the correlation coefficient matrix of different feature maps in the same
layer, named CCM-loss. This loss term can encourage the neural network to learn
stronger linear representation relations between feature maps during the
training from the scratch so that more homogenous parts can be removed later in
pruning. CCM-loss provides us with another universal transcendental
mathematical tool besides L*-norm regularization, which concentrates on
generating zeros, to generate more redundancy but for the different genres.
Furthermore, we design a matching channel selection strategy based on principal
components analysis to exploit the maximum potential ability of CCM-loss. In
our new strategy, we mainly focus on the consistency and integrality of the
information flow in the network. Instead of empirically hard-code the retain
ratio for each layer, our channel selection strategy can dynamically adjust
each layer's retain ratio according to the specific circumstance of a
per-trained model to push the prune ratio to the limit. Notably, on the
Cifar-10 dataset, our method brings 93.64% accuracy for pruned VGG-16 with only
1.40M parameters and 49.60M FLOPs, the pruned ratios for parameters and FLOPs
are 90.6% and 84.2%, respectively. For ResNet-50 trained on the ImageNet
dataset, our approach achieves 42.8% and 47.3% storage and computation
reductions, respectively, with an accuracy of 76.23%. Our code is available at
this https URL

提出了一种新的结构修剪方法，在训练期间利用 CCM-loss 来促使神经网络学习强大的线性表示关系，然后设计了一种基于主成分分析的匹配通道选择策略来最大限度地利用 CCM-loss 的潜在能力，达到更高的剪枝率和减少计算量。

增强线性表示冗余下的卷积神经网络滤波器剪枝

Filter Pruning For CNN With Enhanced Linear Representation Redundancy

Structured network pruning is a practical approach to reduce computation cost
directly while retaining the CNNs' generalization performance in real
applications. However, identifying redundant filters is a core problem in
structured network pruning, and current redundancy criteria only focus on
individual filters' attributes. When pruning sparsity increases, these
redundancy criteria are not effective or efficient enough. Since the
filter-wise interaction also contributes to the CNN's prediction accuracy, we
integrate the filter-wise interaction into the redundancy criterion. In our
criterion, we introduce the filter importance and filter utilization strength
to reflect the decision ability of individual and multiple filters. Utilizing
this new redundancy criterion, we propose a structured network pruning approach
SNPFI (Structured Network Pruning by measuring Filter-wise Interaction). During
the pruning, the SNPFI can automatically assign the proper sparsity based on
the filter utilization strength and eliminate the useless filters by filter
importance. After the pruning, the SNPFI can recover pruned model's performance
effectively without iterative training by minimizing the interaction
difference. We empirically demonstrate the effectiveness of the SNPFI with
several commonly used CNN models, including AlexNet, MobileNetv1, and
ResNet-50, on various image classification datasets, including MNIST, CIFAR-10,
and ImageNet. For all experimental CNN models, nearly 60% of computation is
reduced in a network compression while the classification accuracy remains.

提出了一种新的结构化网络修剪方法 SNPFI，该方法通过衡量滤波器的重要性和利用强度来减少冗余滤波器，并将滤波器间的相互作用纳入重要性度量中，实现减少计算成本并恢复性能的效果，并在各种图像分类数据集上成功验证了该方法的有效性。