In this paper, we explore a principal way to enhance the quality of object
masks produced by different segmentation models. We propose a model-agnostic
solution called SegRefiner, which offers a novel perspective on this problem by
interpreting segmentation refinement as a data generation process. As a result,
the refinement process can be smoothly implemented through a series of
denoising diffusion steps. Specifically, SegRefiner takes coarse masks as
inputs and refines them using a discrete diffusion process. By predicting the
label and corresponding states-transition probabilities for each pixel,
SegRefiner progressively refines the noisy masks in a conditional denoising
manner. To assess the effectiveness of SegRefiner, we conduct comprehensive
experiments on various segmentation tasks, including semantic segmentation,
instance segmentation, and dichotomous image segmentation. The results
demonstrate the superiority of our SegRefiner from multiple aspects. Firstly,
it consistently improves both the segmentation metrics and boundary metrics
across different types of coarse masks. Secondly, it outperforms previous
model-agnostic refinement methods by a significant margin. Lastly, it exhibits
a strong capability to capture extremely fine details when refining
high-resolution images. The source code and trained models are available at
this https URL

提出了 SegRefiner 方法，通过将细分改进视为数据生成过程，通过一系列去噪扩散步骤实现。经过全面实验验证，SegRefiner 在各种细分任务中表现出优越性能，包括语义分割、实例分割和二分图像分割。

SegRefiner：离散扩散过程下的面向模型无关分割精化

SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete  Diffusion Process

Efficient and accurate segmentation of unseen objects is crucial for robotic
manipulation. However, it remains challenging due to over- or
under-segmentation. Although existing refinement methods can enhance the
segmentation quality, they fix only minor boundary errors or are not
sufficiently fast. In this work, we propose INSTAnce Boundary Explicit Error
Estimation and Refinement (INSTA-BEEER), a novel refinement model that allows
for adding and deleting instances and sharpening boundaries. Leveraging an
error-estimation-then-refinement scheme, the model first estimates the
pixel-wise boundary explicit errors: true positive, true negative, false
positive, and false negative pixels of the instance boundary in the initial
segmentation. It then refines the initial segmentation using these error
estimates as guidance. Experiments show that the proposed model significantly
enhances segmentation, achieving state-of-the-art performance. Furthermore,
with a fast runtime (less than 0.1 s), the model consistently improves
performance across various initial segmentation methods, making it highly
suitable for practical robotic applications.

本文提出了一种名为 INSTA-BEEER 的新型细分模型，它采用错误估计然后细化的结构来提高图像分割质量，从而实现在机器人操作等实际应用中高效而精确地分割出未见过的对象。