Machine learning pipelines for classification tasks often train a universal
model to achieve accuracy across a broad range of classes. However, a typical
user encounters only a limited selection of classes regularly. This disparity
provides an opportunity to enhance computational efficiency by tailoring models
to focus on user-specific classes. Existing works rely on unstructured pruning,
which introduces randomly distributed non-zero values in the model, making it
unsuitable for hardware acceleration. Alternatively, some approaches employ
structured pruning, such as channel pruning, but these tend to provide only
minimal compression and may lead to reduced model accuracy. In this work, we
propose CRISP, a novel pruning framework leveraging a hybrid structured
sparsity pattern that combines both fine-grained N:M structured sparsity and
coarse-grained block sparsity. Our pruning strategy is guided by a
gradient-based class-aware saliency score, allowing us to retain weights
crucial for user-specific classes. CRISP achieves high accuracy with minimal
memory consumption for popular models like ResNet-50, VGG-16, and MobileNetV2
on ImageNet and CIFAR-100 datasets. Moreover, CRISP delivers up to 14$\times$
reduction in latency and energy consumption compared to existing pruning
methods while maintaining comparable accuracy. Our code is available at
this https URL

该研究提出了一种名为 CRISP 的修剪框架，利用混合结构稀疏模式，结合精细的 N:M 结构稀疏和粗粒度的块状稀疏，通过基于梯度的类感知显著性分数指导修剪策略，以实现对用户特定类别至关重要的权重保留，从而在 ImageNet 和 CIFAR-100 数据集上实现了高精度和最小内存消耗，并与现有修剪方法相比，延迟和能源消耗降低了高达 14 倍，同时保持了可比较的准确性。