Access to pre-trained models has recently emerged as a standard across
numerous machine learning domains. Unfortunately, access to the original data
the models were trained on may not equally be granted. This makes it
tremendously challenging to fine-tune, compress models, adapt continually, or
to do any other type of data-driven update. We posit that original data access
may however not be required. Specifically, we propose Contrastive Abductive
Knowledge Extraction (CAKE), a model-agnostic knowledge distillation procedure
that mimics deep classifiers without access to the original data. To this end,
CAKE generates pairs of noisy synthetic samples and diffuses them contrastively
toward a model's decision boundary. We empirically corroborate CAKE's
effectiveness using several benchmark datasets and various architectural
choices, paving the way for broad application.

通过噪声合成样本并经过对比扩散，CAKE 不需要原始数据即可模拟深度分类器，从而进行知识蒸馏。

无需访问数据的深度分类器模仿

Deep Classifier Mimicry without Data Access

Filter pruning is widely adopted to compress and accelerate the Convolutional
Neural Networks (CNNs), but most previous works ignore the relationship between
filters and channels in different layers. Processing each layer independently
fails to utilize the collaborative relationship across layers. In this paper,
we intuitively propose a novel pruning method by explicitly leveraging the
Filters Similarity in Consecutive Layers (FSCL). FSCL compresses models by
pruning filters whose corresponding features are more worthless in the model.
The extensive experiments demonstrate the effectiveness of FSCL, and it yields
remarkable improvement over state-of-the-art on accuracy, FLOPs and parameter
reduction on several benchmark models and datasets.

本研究介绍了一种新的滤波器修剪方法（FSCL）， 它在连续层之间明确利用滤波器之间的相似性来压缩模型，从而剪掉那些在模型中不太重要的特征， 并在多个基准模型和数据集上取得了显着的精度、FLOPs 和参数量减少。