Pruning methods have shown to be effective at reducing the size of deep
neural networks while keeping accuracy almost intact. Among the most effective
methods are those that prune a network while training it with a sparsity prior
loss and learnable dropout parameters. A shortcoming of these approaches
however is that neither the size nor the inference speed of the pruned network
can be controlled directly; yet this is a key feature for targeting deployment
of CNNs on low-power hardware. To overcome this, we introduce a budgeted
regularized pruning framework for deep CNNs. Our approach naturally fits into
traditional neural network training as it consists of a learnable masking
layer, a novel budget-aware objective function, and the use of knowledge
distillation. We also provide insights on how to prune a residual network and
how this can lead to new architectures. Experimental results reveal that CNNs
pruned with our method are more accurate and less compute-hungry than
state-of-the-art methods. Also, our approach is more effective at preventing
accuracy collapse in case of severe pruning; this allows us to attain pruning
factors up to 16x without significant accuracy drop.

本研究提出了一种基于预算限制的深度卷积神经网络修剪框架，并利用学习掩码层、创新的预算目标函数以及知识蒸馏等技术进行修剪，实验结果表明，使用我们的方法修剪后的 CNNs 比现有方法更准确、计算更少，同时即使极度剪枝也不影响分类准确度，具有很大的实用价值。