Neural network pruning is an essential approach for reducing the
computational complexity of deep models so that they can be well deployed on
resource-limited devices. Compared with conventional methods, the recently
developed dynamic pruning methods determine redundant filters variant to each
input instance which achieves higher acceleration. Most of the existing methods
discover effective sub-networks for each instance independently and do not
utilize the relationship between different inputs. To maximally excavate
redundancy in the given network architecture, this paper proposes a new
paradigm that dynamically removes redundant filters by embedding the manifold
information of all instances into the space of pruned networks (dubbed as
ManiDP). We first investigate the recognition complexity and feature similarity
between images in the training set. Then, the manifold relationship between
instances and the pruned sub-networks will be aligned in the training
procedure. The effectiveness of the proposed method is verified on several
benchmarks, which shows better performance in terms of both accuracy and
computational cost compared to the state-of-the-art methods. For example, our
method can reduce 55.3% FLOPs of ResNet-34 with only 0.57% top-1 accuracy
degradation on ImageNet.

本论文提出了一种新的范式，通过将所有实例的流形信息嵌入到裁剪网络的空间中 (称为 ManiDP)，以动态地去除冗余的过滤器以最大化挖掘给定网络架构中的冗余。在几个基准测试中验证了所提出的方法的有效性，在精度和计算成本方面显示出与现有技术方法相比更好的性能可将 ResNet-34 的 FLOP 降低 55.3％，且仅仅减少 0.57％的 Top-1 精度，ImageNet。