Network pruning techniques, including weight pruning and filter pruning,
reveal that most state-of-the-art neural networks can be accelerated without a
significant performance drop. This work focuses on filter pruning which enables
accelerated inference with any off-the-shelf deep learning library and
hardware. We propose the concept of \emph{network pruning spaces} that
parametrize populations of subnetwork architectures. Based on this concept, we
explore the structure aspect of subnetworks that result in minimal loss of
accuracy in different pruning regimes and arrive at a series of observations by
comparing subnetwork distributions. We conjecture through empirical studies
that there exists an optimal FLOPs-to-parameter-bucket ratio related to the
design of original network in a pruning regime. Statistically, the structure of
a winning subnetwork guarantees an approximately optimal ratio in this regime.
Upon our conjectures, we further refine the initial pruning space to reduce the
cost of searching a good subnetwork architecture. Our experimental results on
ImageNet show that the subnetwork we found is superior to those from the
state-of-the-art pruning methods under comparable FLOPs.

本研究提出网络剪枝空间的概念，探讨子网络结构在不同剪枝范围内的最小精度损失并证明了在某个剪枝范围内存在最佳的 FLOPs-to-parameter-bucket 比率，通过实验结果表明，我们找到的子网络在合理的 FLOPs 下优于现有最先进的剪枝方法。