The state-of-art DNN structures involve high computation and great demand for memory storage which pose intensive challenge on DNN framework resources. To mitigate the challenges, weight pruning techniques has been studied. However, high accuracy solution for extreme structured pruning that combines different types of structured sparsity still waiting for unraveling due to the extremely reduced weights in DNN networks. In this paper, we propose a DNN framework which combines two different types of structured weight pruning (filter and column prune) by incorporating alternating direction method of multipliers (ADMM) algorithm for better prune performance. We are the first to find non-optimality of ADMM process and unused weights in a structured pruned model, and further design an optimization framework which contains the first proposed Network Purification and Unused Path Removal algorithms which are dedicated to post-processing an structured pruned model after ADMM steps. Some high lights shows we achieve 232x compression on LeNet-5, 60x compression on ResNet-18 CIFAR-10 and over 5x compression on AlexNet. We share our models at anonymous link http://bit.ly/2VJ5ktv.

提出一种融合ADMM算法和滤波器和列剪枝技术的DNN框架，包括网络净化和未使用通路去除算法，可在LeNet-5、ResNet-18 CIFAR-10和AlexNet上实现232倍、60倍和5倍的压缩，以减少DNN框架资源的高计算和大内存存储的挑战。

ResNet裁剪60倍：引入网络净化和剔除未使用路径（P-RM）在权重裁剪之后