Visual concept discovery has long been deemed important to improve
interpretability of neural networks, because a bank of semantically meaningful
concepts would provide us with a starting point for building machine learning
models that exhibit intelligible reasoning process. Previous methods have
disadvantages: either they rely on labelled support sets that incorporate human
biases for objects that are "useful," or they fail to identify multiple
concepts that occur within a single image. We reframe the concept discovery
task as an unsupervised semantic segmentation problem, and present SegDiscover,
a novel framework that discovers semantically meaningful visual concepts from
imagery datasets with complex scenes without supervision. Our method contains
three important pieces: generating concept primitives from raw images,
discovering concepts by clustering in the latent space of a self-supervised
pretrained encoder, and concept refinement via neural network smoothing.
Experimental results provide evidence that our method can discover multiple
concepts within a single image and outperforms state-of-the-art unsupervised
methods on complex datasets such as Cityscapes and COCO-Stuff. Our method can
be further used as a neural network explanation tool by comparing results
obtained by different encoders.

SegDiscover 是一个无人监督语义分割框架，通过从原始图像生成概念原语、在自监督预训练编码器的潜在空间中进行聚类以及神经网络平滑来发现语义上有意义的视觉概念。实验结果表明，该方法可以在单个图像中发现多个概念，并在包括 Cityscapes 和 COCO-Stuff 等复杂数据集上优于最先进的无监督方法。此方法还可以作为神经网络解释工具使用，通过比较不同编码器得出结果。