We present a novel usage of Transformers to make image classification
interpretable. Unlike mainstream classifiers that wait until the last
fully-connected layer to incorporate class information to make predictions, we
investigate a proactive approach, asking each class to search for itself in an
image. We realize this idea via a Transformer encoder-decoder inspired by
DEtection TRansformer (DETR). We learn ``class-specific'' queries (one for each
class) as input to the decoder, enabling each class to localize its patterns in
an image via cross-attention. We name our approach INterpretable TRansformer
(INTR), which is fairly easy to implement and exhibits several compelling
properties. We show that INTR intrinsically encourages each class to attend
distinctively; the cross-attention weights thus provide a faithful
interpretation of the prediction. Interestingly, via ``multi-head''
cross-attention, INTR could identify different ``attributes'' of a class,
making it particularly suitable for fine-grained classification and analysis,
which we demonstrate on eight datasets. Our code and pre-trained model are
publicly accessible at this https URL

我们提出了一种新颖的 Transformer 的用法，使图像分类具有解释性。通过在编码器 - 解码器中使用 Transformer 进行类别定位，我们的方法 INTR 使得每个类别都能够在图像中找到自己的模式，并展示了对预测的准确解释。通过多头交叉关注机制，INTR 能够识别类别的不同属性，从而在细粒度分类和分析方面具有良好的适应性。