Self-supervised pre-training strategies have recently shown impressive results for training general-purpose feature extraction backbones in computer vision. In combination with the Vision Transformer architecture, the DINO self-distillation technique has interesting emerging properties, such as unsupervised clustering in the latent space and semantic correspondences of the produced features without using explicit human-annotated labels. The STEGO method for unsupervised semantic segmentation contrastively distills feature correspondences of a DINO-pre-trained Vision Transformer and recently set a new state of the art. However, the detailed workings of STEGO have yet to be disentangled, preventing its usage in safety-critical applications. This paper provides a deeper understanding of the STEGO architecture and training strategy by conducting studies that uncover the working mechanisms behind STEGO, reproduce and extend its experimental validation, and investigate the ability of STEGO to transfer to different datasets. Results demonstrate that the STEGO architecture can be interpreted as a semantics-preserving dimensionality reduction technique.

本文研究使用自监督预训练策略和Vision Transformer架构的DINO自蒸馏技术，以无监督聚类和语义对应的方式进行通用特征提取；并介绍了基于DINO预训练的Vision Transformer的特征相应的无监督语义分割的STEGO方法，并探讨了STEGO方法的工作机制和训练策略，最终得出STEGO构架可被解释为保留语义的降维技术，是当前技术应用于安全关键型应用的一种方法。

揭示 STEGO 的内部工作机理，以实现安全的无监督语义分割