This paper proposes a novel zero-shot edge detection with SCESAME, which
stands for Spectral Clustering-based Ensemble for Segment Anything Model
Estimation, based on the recently proposed Segment Anything Model (SAM). SAM is
a foundation model for segmentation tasks, and one of the interesting
applications of SAM is Automatic Mask Generation (AMG), which generates
zero-shot segmentation masks of an entire image. AMG can be applied to edge
detection, but suffers from the problem of overdetecting edges. Edge detection
with SCESAME overcomes this problem by three steps: (1) eliminating small
generated masks, (2) combining masks by spectral clustering, taking into
account mask positions and overlaps, and (3) removing artifacts after edge
detection. We performed edge detection experiments on two datasets, BSDS500 and
NYUDv2. Although our zero-shot approach is simple, the experimental results on
BSDS500 showed almost identical performance to human performance and CNN-based
methods from seven years ago. In the NYUDv2 experiments, it performed almost as
well as recent CNN-based methods. These results indicate that our method has
the potential to be a strong baseline for future zero-shot edge detection
methods. Furthermore, SCESAME is not only applicable to edge detection, but
also to other downstream zero-shot tasks.

提出了一种新颖的零样本边缘检测方法 SCESAME, 基于最近提出的 Segment Anything Model (SAM). 结果表明，SCESAME 方法在 BSDS500 数据集上表现几乎与人类和基于卷积神经网络的方法相同，在 NYUDv2 数据集上表现几乎与最近的基于卷积神经网络的方法相当，表明其具有成为将来零样本边缘检测方法的坚实基础的潜力。

零样本边缘检测：基于 SCESAME 的谱聚类集成模型估计

Zero-Shot Edge Detection with SCESAME: Spectral Clustering-based  Ensemble for Segment Anything Model Estimation

The objective of the image inpainting task is to fill missing regions of an
image in a visually plausible way. Recently, deep-learning-based image
inpainting networks have generated outstanding results, and some utilize their
models as object removers by masking unwanted objects in an image. However,
while trying to better remove objects using their networks, the previous works
pay less attention to the importance of the input mask. In this paper, we focus
on generating the input mask to better remove objects using the off-the-shelf
image inpainting network. We propose an automatic mask generator inspired by
the explainable AI (XAI) method, whose output can better remove objects than a
semantic segmentation mask. The proposed method generates an importance map
using randomly sampled input masks and quantitatively estimated scores of the
completed images obtained from the random masks. The output mask is selected by
a judge module among the candidate masks which are generated from the
importance map. We design the judge module to quantitatively estimate the
quality of the object removal results. In addition, we empirically find that
the evaluation methods used in the previous works reporting object removal
results are not appropriate for estimating the performance of an object
remover. Therefore, we propose new evaluation metrics (FID$^*$ and U-IDS$^*$)
to properly evaluate the quality of object removers. Experiments confirm that
our method shows better performance in removing target class objects than the
masks generated from the semantic segmentation maps, and the two proposed
metrics make judgments consistent with humans.

该论文提出了一种基于可解释的人工智能 (XAI) 方法的自动输入遮罩生成器，可以更好地移除图像中的物体，并提出了 FID$^*$ 和 U-IDS$^*$ 等新的评估指标来适当评估物体移除器的质量。