In this paper, we propose a novel layer-adaptive weight-pruning approach for
Deep Neural Networks (DNNs) that addresses the challenge of optimizing the
output distortion minimization while adhering to a target pruning ratio
constraint. Our approach takes into account the collective influence of all
layers to design a layer-adaptive pruning scheme. We discover and utilize a
very important additivity property of output distortion caused by pruning
weights on multiple layers. This property enables us to formulate the pruning
as a combinatorial optimization problem and efficiently solve it through
dynamic programming. By decomposing the problem into sub-problems, we achieve
linear time complexity, making our optimization algorithm fast and feasible to
run on CPUs. Our extensive experiments demonstrate the superiority of our
approach over existing methods on the ImageNet and CIFAR-10 datasets. On
CIFAR-10, our method achieves remarkable improvements, outperforming others by
up to 1.0% for ResNet-32, 0.5% for VGG-16, and 0.7% for DenseNet-121 in terms
of top-1 accuracy. On ImageNet, we achieve up to 4.7% and 4.6% higher top-1
accuracy compared to other methods for VGG-16 and ResNet-50, respectively.
These results highlight the effectiveness and practicality of our approach for
enhancing DNN performance through layer-adaptive weight pruning. Code will be
available on this https URL

我们提出了一种新颖的层自适应权重剪枝方法，通过优化输出失真最小化并同时遵守目标剪枝比例约束，实现了对深度神经网络 (DNNs) 的优化。我们发现和利用了多个层的权重剪枝导致的输出失真具有非常重要的可加性特性，并基于此特性将剪枝问题建模为一个组合优化问题，并通过动态规划高效解决。我们用子问题分解的方式使问题具有线性时间复杂度，从而使我们的优化算法快速且适用于 CPU 运行。在 ImageNet 和 CIFAR-10 数据集上的广泛实验证明了我们方法的优越性。在 CIFAR-10 上，我们的方法在 ResNet-32、VGG-16 和 DenseNet-121 的 top-1 准确率方面分别超过了其他方法 1.0%、0.5% 和 0.7%。在 ImageNet 上，我们的方法分别比其他方法提高了 VGG-16 和 ResNet-50 的 top-1 准确率高达 4.7% 和 4.6%。这些结果突出了我们的方法通过层自适应权重剪枝来提升 DNN 性能的效果和实用性。代码将在此 https URL 上提供。