Diffusion models have demonstrated excellent performance in image generation.
Although various few-shot semantic segmentation (FSS) models with different
network structures have been proposed, performance improvement has reached a
bottleneck. This paper presents the first work to leverage the diffusion model
for FSS task, called DifFSS. DifFSS, a novel FSS paradigm, can further improve
the performance of the state-of-the-art FSS models by a large margin without
modifying their network structure. Specifically, we utilize the powerful
generation ability of diffusion models to generate diverse auxiliary support
images by using the semantic mask, scribble or soft HED boundary of the support
image as control conditions. This generation process simulates the variety
within the class of the query image, such as color, texture variation,
lighting, $etc$. As a result, FSS models can refer to more diverse support
images, yielding more robust representations, thereby achieving a consistent
improvement in segmentation performance. Extensive experiments on three
publicly available datasets based on existing advanced FSS models demonstrate
the effectiveness of the diffusion model for FSS task. Furthermore, we explore
in detail the impact of different input settings of the diffusion model on
segmentation performance. Hopefully, this completely new paradigm will bring
inspiration to the study of FSS task integrated with AI-generated content.

DifFSS is a novel few-shot semantic segmentation paradigm which leverages the powerful generation ability of diffusion models to generate diverse auxiliary support images for FSS models by using semantic masks, scribbles or soft HED boundaries without modifying their network structure, leading to a consistent improvement in segmentation performance.