The lack of interpretability of the Vision Transformer may hinder its use in critical real-world applications despite its effectiveness. To overcome this issue, we propose a post-hoc interpretability method called VISION DIFFMASK, which uses the activations of the model's hidden layers to predict the relevant parts of the input that contribute to its final predictions. Our approach uses a gating mechanism to identify the minimal subset of the original input that preserves the predicted distribution over classes. We demonstrate the faithfulness of our method, by introducing a faithfulness task, and comparing it to other state-of-the-art attribution methods on CIFAR-10 and ImageNet-1K, achieving compelling results. To aid reproducibility and further extension of our work, we open source our implementation: https://github.com/AngelosNal/Vision-DiffMask

提出了一种名为VISION DIFFMASK的后处理可解释性方法，使用模型隐藏层的激活来预测对其最终预测有贡献的输入的相关部分，并使用门控机制识别原始输入的最小子集，以维持对类的预测分布的预测结果；结果表明该方法在CIFAR-10和ImageNet-1K上比其他最先进的归因方法更为可靠。

VISION DIFFMASK：可微分patch掩模的视觉transformer忠实解释