With the remarkable success of deep learning recently, efficient network
compression algorithms are urgently demanded for releasing the potential
computational power of edge devices, such as smartphones or tablets. However,
optimal network pruning is a non-trivial task which mathematically is an
NP-hard problem. Previous researchers explain training a pruned network as
buying a lottery ticket. In this paper, we investigate the Magnitude-Based
Pruning (MBP) scheme and analyze it from a novel perspective through Fourier
analysis on the deep learning model to guide model designation. Besides
explaining the generalization ability of MBP using Fourier transform, we also
propose a novel two-stage pruning approach, where one stage is to obtain the
topological structure of the pruned network and the other stage is to retrain
the pruned network to recover the capacity using knowledge distillation from
lower to higher on the frequency domain. Extensive experiments on CIFAR-10 and
CIFAR-100 demonstrate the superiority of our novel Fourier analysis based MBP
compared to other traditional MBP algorithms.

本文探讨基于傅里叶分析的 Magnitude-Based Pruning (MBP) 算法，在深度学习模型中进行网络压缩和模型设计，并提出了一种新颖的两阶段剪枝方法。实验结果表明，我们提出的基于傅里叶分析的 MBP 算法相比其他传统 MBP 算法具有优越性。