The rapid increase in the parameters of deep learning models has led to
significant costs, challenging computational efficiency and model
interpretability. In this paper, we introduce a novel and straightforward
neural network pruning framework that incorporates the Gumbel-Softmax
technique. This framework enables the simultaneous optimization of a network's
weights and topology in an end-to-end process using stochastic gradient
descent. Empirical results demonstrate its exceptional compression capability,
maintaining high accuracy on the MNIST dataset with only 0.15\% of the original
network parameters. Moreover, our framework enhances neural network
interpretability, not only by allowing easy extraction of feature importance
directly from the pruned network but also by enabling visualization of feature
symmetry and the pathways of information propagation from features to outcomes.
Although the pruning strategy is learned through deep learning, it is
surprisingly intuitive and understandable, focusing on selecting key
representative features and exploiting data patterns to achieve extreme sparse
pruning. We believe our method opens a promising new avenue for deep learning
pruning and the creation of interpretable machine learning systems.

我们介绍了一种新颖而简单的神经网络剪枝框架，通过引入 Gumbel-Softmax 技术，在端到端的过程中使用随机梯度下降同时优化网络的权重和拓扑结构，实现了网络的卓越压缩能力，保持了在 MNIST 数据集上的高准确性，仅使用了原始网络参数的 0.15％。此外，我们的框架提升了神经网络的可解释性，不仅能够轻松从剪枝网络中提取特征重要性，而且能够可视化特征对称性和信息传递路径，学习策略通过深度学习得到，但却令人惊讶地直观且可理解，专注于选择重要的代表性特征并利用数据模式实现极度稀疏的剪枝。我们相信我们的方法为深度学习剪枝和可解释的机器学习系统的创建开辟了有前途的新途径。