Neural network sparsity has attracted many research interests due to its
similarity to biological schemes and high energy efficiency. However, existing
methods depend on long-time training or fine-tuning, which prevents large-scale
applications. Recently, some works focusing on post-training sparsity (PTS)
have emerged. They get rid of the high training cost but usually suffer from
distinct accuracy degradation due to neglect of the reasonable sparsity rate at
each layer. Previous methods for finding sparsity rates mainly focus on the
training-aware scenario, which usually fails to converge stably under the PTS
setting with limited data and much less training cost. In this paper, we
propose a fast and controllable post-training sparsity (FCPTS) framework. By
incorporating a differentiable bridge function and a controllable optimization
objective, our method allows for rapid and accurate sparsity allocation
learning in minutes, with the added assurance of convergence to a predetermined
global sparsity rate. Equipped with these techniques, we can surpass the
state-of-the-art methods by a large margin, e.g., over 30\% improvement for
ResNet-50 on ImageNet under the sparsity rate of 80\%. Our plug-and-play code
and supplementary materials are open-sourced at
this https URL

通过结合可微的桥接函数和可控的优化目标，我们提出了一种快速且可控的后向训练稀疏性（FCPTS）框架，使得在几分钟内可以实现快速和准确的稀疏性分配学习，并保证收敛到预定的全局稀疏率。利用这些技术，我们能够大幅超越现有的方法，在 80% 的稀疏率下，ResNet-50 在 ImageNet 上取得超过 30% 的改进。