Recent discoveries on neural network pruning reveal that, with a carefully
chosen layerwise sparsity, a simple magnitude-based pruning achieves
state-of-the-art tradeoff between sparsity and performance. However, without a
clear consensus on "how to choose," the layerwise sparsities are mostly
selected algorithm-by-algorithm, often resorting to handcrafted heuristics or
an extensive hyperparameter search. To fill this gap, we propose a novel
importance score for global pruning, coined layer-adaptive magnitude-based
pruning (LAMP) score; the score is a rescaled version of weight magnitude that
incorporates the model-level $\ell_2$ distortion incurred by pruning, and does
not require any hyperparameter tuning or heavy computation. Under various image
classification setups, LAMP consistently outperforms popular existing schemes
for layerwise sparsity selection. Furthermore, we observe that LAMP continues
to outperform baselines even in weight-rewinding setups, while the
connectivity-oriented layerwise sparsity (the strongest baseline overall)
performs worse than a simple global magnitude-based pruning in this case. Code:
this https URL

这篇论文针对神经网络修剪方法提出一种基于重要性分数的全局修剪方法（LAMP），不需要人工调参或加重的计算，使用在图像分类任务上表现出优于现有算法的效果。