Deep neural networks (DNN) usually come with a significant computational
burden. While approaches such as structured pruning and mobile-specific DNNs
have been proposed, they incur drastic accuracy loss. In this paper we leverage
the intrinsic redundancy in latent representations to reduce the computational
load with limited loss in performance. We show that semantically similar inputs
share many filters, especially in the earlier layers. Thus, semantically
similar classes can be clustered to create cluster-specific subgraphs. To this
end, we propose a new framework called Semantic Inference (SINF). In short,
SINF (i) identifies the semantic cluster the object belongs to using a small
additional classifier and (ii) executes the subgraph extracted from the base
DNN related to that semantic cluster for inference. To extract each
cluster-specific subgraph, we propose a new approach named Discriminative
Capability Score (DCS) that finds the subgraph with the capability to
discriminate among the members of a specific semantic cluster. DCS is
independent from SINF and can be applied to any DNN. We benchmark the
performance of DCS on the VGG16, VGG19, and ResNet50 DNNs trained on the
CIFAR100 dataset against 6 state-of-the-art pruning approaches. Our results
show that (i) SINF reduces the inference time of VGG19, VGG16, and ResNet50
respectively by up to 35%, 29% and 15% with only 0.17%, 3.75%, and 6.75%
accuracy loss (ii) DCS achieves respectively up to 3.65%, 4.25%, and 2.36%
better accuracy with VGG16, VGG19, and ResNet50 with respect to existing
discriminative scores (iii) when used as a pruning criterion, DCS achieves up
to 8.13% accuracy gain with 5.82% less parameters than the existing state of
the art work published at ICLR 2023 (iv) when considering per-cluster accuracy,
SINF performs on average 5.73%, 8.38% and 6.36% better than the base VGG16,
VGG19, and ResNet50.

通过利用潜在表示中的内在冗余来减少计算负担，我们提出了一种名为语义推理（SINF）的新框架，它可以识别物体属于的语义簇并执行与该语义簇相关的子图进行推理，并利用名为判别能力分数（DCS）的方法提取每个簇特定的子图。在基准测试中，我们的结果表明，SINF 可以减少 VGG19、VGG16 和 ResNet50 的推理时间，同时只损失 0.17%、3.75% 和 6.75% 的精度；DCS 在 VGG16、VGG19 和 ResNet50 中取得优秀的精度，比现有的判别分数高出分别为 3.65%、4.25% 和 2.36%；DCS 作为剪枝标准，与已有的 ICLR2023 最新工作相比，可以提高 8.13% 的准确率，并减少 5.82% 的参数量；SINF 在考虑每个簇的准确率时，相对于 VGG16、VGG19 和 ResNet50 的基础模型，性能提高了平均分别为 5.73%、8.38% 和 6.36%。