Current structured pruning methods often result in considerable accuracy
drops due to abrupt network changes and loss of information from pruned
structures. To address these issues, we introduce the Decay Pruning Method
(DPM), a novel smooth pruning approach with a self-rectifying mechanism. DPM
consists of two key components: (i) Smooth Pruning: It converts conventional
single-step pruning into multi-step smooth pruning, gradually reducing
redundant structures to zero over N steps with ongoing optimization. (ii)
Self-Rectifying: This procedure further enhances the aforementioned process by
rectifying sub-optimal pruning based on gradient information. Our approach
demonstrates strong generalizability and can be easily integrated with various
existing pruning methods. We validate the effectiveness of DPM by integrating
it with three popular pruning methods: OTOv2, Depgraph, and Gate Decorator.
Experimental results show consistent improvements in performance compared to
the original pruning methods, along with further reductions of FLOPs in most
scenarios.

当前的结构化剪枝方法常常由于网络的突变和剪枝结构的信息丢失导致准确率显著下降。为了解决这些问题，我们引入了 Decay Pruning Method (DPM)，一种具有自校正机制的新型平滑剪枝方法。DPM 包括两个关键组成部分：(i) 平滑剪枝：将传统的单步剪枝转化为多步平滑剪枝，在持续优化中逐步将冗余结构减少到零。(ii) 自校正：该过程通过基于梯度信息纠正亚最优剪枝，进一步增强了前述过程。我们的方法表现出了很强的通用性，可以轻松与各种现有的剪枝方法相结合。我们将其与三种流行的剪枝方法 OTOv2、Depgraph 和 Gate Decorator 集成，实验证明与原始剪枝方法相比，性能有持续的改善，并在大多数情况下进一步降低了 FLOPs。