Discovering a high-performing sparse network within a massive neural network
is advantageous for deploying them on devices with limited storage, such as
mobile phones. Additionally, model explainability is essential to fostering
trust in AI. The Lottery Ticket Hypothesis (LTH) finds a network within a deep
network with comparable or superior performance to the original model. However,
limited study has been conducted on the success or failure of LTH in terms of
explainability. In this work, we examine why the performance of the pruned
networks gradually increases or decreases. Using Grad-CAM and Post-hoc concept
bottleneck models (PCBMs), respectively, we investigate the explainability of
pruned networks in terms of pixels and high-level concepts. We perform
extensive experiments across vision and medical imaging datasets. As more
weights are pruned, the performance of the network degrades. The discovered
concepts and pixels from the pruned networks are inconsistent with the original
network -- a possible reason for the drop in performance.

在本研究中，我们通过使用 Grad-CAM 和 Post-hoc 概念瓶颈模型（PCBMs）分别从像素和高级概念的角度，调查了经过裁剪的网络的可解释性。通过在视觉和医学影像数据集上进行广泛实验，我们发现随着权重的减少，网络的性能会下降，并且从裁剪网络中发现的概念和像素与原始网络不一致，可能是性能下降的原因。