Recent developments in self-supervised learning (SSL) have made it possible
to learn data representations without the need for annotations. Inspired by the
non-contrastive SSL approach (SimSiam), we introduce a novel framework SIMSAM
to compute the Semantic Affinity Matrix, which is significant for unsupervised
image segmentation. Given an image, SIMSAM first extracts features using
pre-trained DINO-ViT, then projects the features to predict the correlations of
dense features in a non-contrastive way. We show applications of the Semantic
Affinity Matrix in object segmentation and semantic segmentation tasks. Our
code is available at this https URL

通过自我监督学习（SSL）的最新发展，我们介绍了一种计算语义亲和矩阵的新框架 SIMSAM，该框架对于无监督图像分割非常重要。

SimSAM：基于简化孪生表示的语义亲和矩阵用于无监督图像分割

SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix  for Unsupervised Image Segmentation

The recently released Segment Anything Model (SAM) has shown powerful
zero-shot segmentation capabilities through a semi-automatic annotation setup
in which the user can provide a prompt in the form of clicks or bounding boxes.
There is growing interest around applying this to medical imaging, where the
cost of obtaining expert annotations is high, privacy restrictions may limit
sharing of patient data, and model generalisation is often poor. However, there
are large amounts of inherent uncertainty in medical images, due to unclear
object boundaries, low-contrast media, and differences in expert labelling
style. Currently, SAM is known to struggle in a zero-shot setting to adequately
annotate the contours of the structure of interest in medical images, where the
uncertainty is often greatest, thus requiring significant manual correction. To
mitigate this, we introduce \textbf{Sim}ulated Interaction for \textbf{S}egment
\textbf{A}nything \textbf{M}odel (\textsc{\textbf{SimSAM}}), an approach that
leverages simulated user interaction to generate an arbitrary number of
candidate masks, and uses a novel aggregation approach to output the most
compatible mask. Crucially, our method can be used during inference directly on
top of SAM, without any additional training requirement. Quantitatively, we
evaluate our method across three publicly available medical imaging datasets,
and find that our approach leads to up to a 15.5\% improvement in contour
segmentation accuracy compared to zero-shot SAM. Our code is available at
https://github.com/BenjaminTowle/SimSAM.

采用模拟用户交互生成候选掩码并使用新的聚合方法输出最兼容的掩码，Simulated Interaction for Segment Anything Model (SimSAM) 在医学图像中展示出较零射击方法有着更高的分割准确性。