Channel pruning and tensor decomposition have received extensive attention in
convolutional neural network compression. However, these two techniques are
traditionally deployed in an isolated manner, leading to significant accuracy
drop when pursuing high compression rates. In this paper, we propose a
Collaborative Compression (CC) scheme, which joints channel pruning and tensor
decomposition to compress CNN models by simultaneously learning the model
sparsity and low-rankness. Specifically, we first investigate the compression
sensitivity of each layer in the network, and then propose a Global Compression
Rate Optimization that transforms the decision problem of compression rate into
an optimization problem. After that, we propose multi-step heuristic
compression to remove redundant compression units step-by-step, which fully
considers the effect of the remaining compression space (i.e., unremoved
compression units). Our method demonstrates superior performance gains over
previous ones on various datasets and backbone architectures. For example, we
achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with
only a Top-1 accuracy drop of 0.56% on ImageNet 2012.

本文提出了一种协同压缩方案，将通道剪枝和张量分解结合起来，通过同时学习模型的稀疏性和低秩性来压缩 CNN 模型。该方法在各个数据集和骨干架构上展示了优越的性能提升，例如在 ImageNet 2012 上，我们通过删除 48.4％的参数从 ResNet-50 模型中达到 52.9％的 FLOPs 减少，仅有 0.56％的 Top-1 准确度降低。