Sparsification-based pruning has been an important category in model compression. Existing methods commonly set sparsity-inducing penalty terms to suppress the importance of dropped weights, which is regarded as the suppressed sparsification paradigm. However, this paradigm inactivates the dropped parts of networks causing capacity damage before pruning, thereby leading to performance degradation. To alleviate this issue, we first study and reveal the relative sparsity effect in emerging stimulative training and then propose a structured pruning framework, named STP, based on an enhanced sparsification paradigm which maintains the magnitude of dropped weights and enhances the expressivity of kept weights by self-distillation. Besides, to find an optimal architecture for the pruned network, we propose a multi-dimension architecture space and a knowledge distillation-guided exploration strategy. To reduce the huge capacity gap of distillation, we propose a subnet mutating expansion technique. Extensive experiments on various benchmarks indicate the effectiveness of STP. Specifically, without fine-tuning, our method consistently achieves superior performance at different budgets, especially under extremely aggressive pruning scenarios, e.g., remaining 95.11% Top-1 accuracy (72.43% in 76.15%) while reducing 85% FLOPs for ResNet-50 on ImageNet. Codes will be released soon.

基于稀疏化剪枝的研究中，我们提出了一种增强稀疏化范式的结构化剪枝框架(STP)，通过自蒸馏的方式维持被剪枝权重的大小并增强保留权重的表现力。此外，为了找到最优的剪枝网络架构，我们采用了多维架构空间和知识蒸馏引导的探索策略，同时使用子网变异扩展技术来减小蒸馏的容量差距。大量实验证明了STP的有效性，特别是在极度激进的剪枝情况下，例如在ImageNet上对ResNet-50进行剪枝，保持95.11%的Top-1准确率（从76.15%减少85%的浮点操作）。

通过刺激训练增强稀疏化