Detecting glass regions is a challenging task due to the ambiguity of their
transparency and reflection properties. These transparent glasses share the
visual appearance of both transmitted arbitrary background scenes and reflected
objects, thus having no fixed patterns.Recent visual foundation models, which
are trained on vast amounts of data, have manifested stunning performance in
terms of image perception and image generation. To segment glass surfaces with
higher accuracy, we make full use of two visual foundation models: Segment
Anything (SAM) and Stable Diffusion.Specifically, we devise a simple glass
surface segmentor named GEM, which only consists of a SAM backbone, a simple
feature pyramid, a discerning query selection module, and a mask decoder. The
discerning query selection can adaptively identify glass surface features,
assigning them as initialized queries in the mask decoder. We also propose a
Synthetic but photorealistic large-scale Glass Surface Detection dataset dubbed
S-GSD via diffusion model with four different scales, which contain 1x, 5x,
10x, and 20x of the original real data size. This dataset is a feasible source
for transfer learning. The scale of synthetic data has positive impacts on
transfer learning, while the improvement will gradually saturate as the amount
of data increases. Extensive experiments demonstrate that GEM achieves a new
state-of-the-art on the GSD-S validation set (IoU +2.1%). Codes and datasets
are available at: this https URL

通过使用视觉基础模型，我们提出了一个名为 GEM 的简单玻璃表面分割器，该分割器能够自适应地识别玻璃表面特征，并在 GSD-S 验证集上达到了新的最佳性能（IoU+2.1%）。