Structured pruning methods are developed to bridge the gap between the
massive scale of neural networks and the limited hardware resources. Most
current structured pruning methods rely on training datasets to fine-tune the
compressed model, resulting in high computational burdens and being
inapplicable for scenarios with stringent requirements on privacy and security.
As an alternative, some data-free methods have been proposed, however, these
methods often require handcraft parameter tuning and can only achieve
inflexible reconstruction. In this paper, we propose the Automatic Data-Free
Pruning (AutoDFP) method that achieves automatic pruning and reconstruction
without fine-tuning. Our approach is based on the assumption that the loss of
information can be partially compensated by retaining focused information from
similar channels. Specifically, We formulate data-free pruning as an
optimization problem, which can be effectively addressed through reinforcement
learning. AutoDFP assesses the similarity of channels for each layer and
provides this information to the reinforcement learning agent, guiding the
pruning and reconstruction process of the network. We evaluate AutoDFP with
multiple networks on multiple datasets, achieving impressive compression
results. For instance, on the CIFAR-10 dataset, AutoDFP demonstrates a 2.87\%
reduction in accuracy loss compared to the recently proposed data-free pruning
method DFPC with fewer FLOPs on VGG-16. Furthermore, on the ImageNet dataset,
AutoDFP achieves 43.17\% higher accuracy than the SOTA method with the same
80\% preserved ratio on MobileNet-V1.

我们提出了一种自动剪枝和重构的无数据方法（AutoDFP），该方法基于强化学习，通过评估每层通道的相似性来指导网络的剪枝和重构过程，并在多个数据集上实现了令人印象深刻的压缩结果。

AutoDFP：通过通道相似重建自动无数据修剪

AutoDFP: Automatic Data-Free Pruning via Channel Similarity  Reconstruction

Structured pruning and quantization are promising approaches for reducing the
inference time and memory footprint of neural networks. However, most existing
methods require the original training dataset to fine-tune the model. This not
only brings heavy resource consumption but also is not possible for
applications with sensitive or proprietary data due to privacy and security
concerns. Therefore, a few data-free methods are proposed to address this
problem, but they perform data-free pruning and quantization separately, which
does not explore the complementarity of pruning and quantization. In this
paper, we propose a novel framework named Unified Data-Free Compression(UDFC),
which performs pruning and quantization simultaneously without any data and
fine-tuning process. Specifically, UDFC starts with the assumption that the
partial information of a damaged(e.g., pruned or quantized) channel can be
preserved by a linear combination of other channels, and then derives the
reconstruction form from the assumption to restore the information loss due to
compression. Finally, we formulate the reconstruction error between the
original network and its compressed network, and theoretically deduce the
closed-form solution. We evaluate the UDFC on the large-scale image
classification task and obtain significant improvements over various network
architectures and compression methods. For example, we achieve a 20.54%
accuracy improvement on ImageNet dataset compared to SOTA method with 30%
pruning ratio and 6-bit quantization on ResNet-34.

提出了一种名为 Unified Data-Free Compression（UDFC）的新框架，它能够在没有数据和微调过程的情况下同时进行修剪和量化，从而减少神经网络的推理时间和内存占用。在大规模图像分类任务上评估 UDFC，与各种网络架构和压缩方法相比，获得了显著的改进。