We study network pruning which aims to remove redundant channels/kernels and
hence speed up the inference of deep networks. Existing pruning methods either
train from scratch with sparsity constraints or minimize the reconstruction
error between the feature maps of the pre-trained models and the compressed
ones. Both strategies suffer from some limitations: the former kind is
computationally expensive and difficult to converge, while the latter kind
optimizes the reconstruction error but ignores the discriminative power of
channels. In this paper, we propose a simple-yet-effective method called
discrimination-aware channel pruning (DCP) to choose the channels that actually
contribute to the discriminative power. Note that a channel often consists of a
set of kernels. Besides the redundancy in channels, some kernels in a channel
may also be redundant and fail to contribute to the discriminative power of the
network, resulting in kernel level redundancy. To solve this, we propose a
discrimination-aware kernel pruning (DKP) method to further compress deep
networks by removing redundant kernels. To prevent DCP/DKP from selecting
redundant channels/kernels, we propose a new adaptive stopping condition, which
helps to automatically determine the number of selected channels/kernels and
often results in more compact models with better performance. Extensive
experiments on both image classification and face recognition demonstrate the
effectiveness of our methods. For example, on ILSVRC-12, the resultant
ResNet-50 model with 30% reduction of channels even outperforms the baseline
model by 0.36% in terms of Top-1 accuracy. The pruned MobileNetV1 and
MobileNetV2 achieve 1.93x and 1.42x inference acceleration on a mobile device,
respectively, with negligible performance degradation. The source code and the
pre-trained models are available at this https URL

该研究介绍了一种简单而有效的方法（称为 DCP），旨在尽可能地选择实际有助于卷积神经网络判别能力的通道，并进一步通过去除多余的内核来压缩深层网络，同时通过自适应停止条件来防止选择冗余的通道 / 内核并实现更好的性能。

针对深度模型压缩的鉴别感知网络裁剪

Discrimination-aware Network Pruning for Deep Model Compression

Channel pruning is one of the predominant approaches for deep model
compression. Existing pruning methods either train from scratch with sparsity
constraints on channels, or minimize the reconstruction error between the
pre-trained feature maps and the compressed ones. Both strategies suffer from
some limitations: the former kind is computationally expensive and difficult to
converge, whilst the latter kind optimizes the reconstruction error but ignores
the discriminative power of channels. To overcome these drawbacks, we
investigate a simple-yet-effective method, called discrimination-aware channel
pruning, to choose those channels that really contribute to discriminative
power. To this end, we introduce additional losses into the network to increase
the discriminative power of intermediate layers and then select the most
discriminative channels for each layer by considering the additional loss and
the reconstruction error. Last, we propose a greedy algorithm to conduct
channel selection and parameter optimization in an iterative way. Extensive
experiments demonstrate the effectiveness of our method. For example, on
ILSVRC-12, our pruned ResNet-50 with 30% reduction of channels even outperforms
the original model by 0.39% in top-1 accuracy.

介绍了一种有效的基于鉴别力的通道剪枝方法，利用额外的损失函数增加中间层的鉴别力并根据重构误差和额外损失函数选择最具鉴别力的通道，并提出了一种迭代贪心算法来优化参数和通道选择。对类似于 ILSVRC-12 的测试数据，使用该算法对 ResNet-50 剪枝显示出相当有效的性能。