Neural networks have achieved remarkable performance in various application
domains. Nevertheless, a large number of weights in pre-trained deep neural
networks prohibit them from being deployed on smartphones and embedded systems.
It is highly desirable to obtain lightweight versions of neural networks for
inference in edge devices. Many cost-effective approaches were proposed to
prune dense and convolutional layers that are common in deep neural networks
and dominant in the parameter space. However, a unified theoretical foundation
for the problem mostly is missing. In this paper, we identify the close
connection between matrix spectrum learning and neural network training for
dense and convolutional layers and argue that weight pruning is essentially a
matrix sparsification process to preserve the spectrum. Based on the analysis,
we also propose a matrix sparsification algorithm tailored for neural network
pruning that yields better pruning result. We carefully design and conduct
experiments to support our arguments. Hence we provide a consolidated viewpoint
for neural network pruning and enhance the interpretability of deep neural
networks by identifying and preserving the critical neural weights.

通过矩阵稀疏化的过程，我们提出了适用于神经网络剪枝的矩阵稀疏化算法，以保留关键神经权重，从而为神经网络剪枝提供一个统一的理论基础，并提高了深度神经网络的可解释性。

神经网络剪枝作为谱保持过程

Neural Network Pruning as Spectrum Preserving Process

We revisit the matrix problems sparse null space and matrix sparsification,
and show that they are equivalent. We then proceed to seek algorithms for these
problems: We prove the hardness of approximation of these problems, and also
give a powerful tool to extend algorithms and heuristics for sparse
approximation theory to these problems.

矩阵问题 —— 稀疏零空间和矩阵稀疏化的问题已被证明是等价的，但其近似难度证明较难且算法需要拓展。