Pre-trained diffusion models have demonstrated remarkable proficiency in
synthesizing images across a wide range of scenarios with customizable prompts,
indicating their effective capacity to capture universal features. Motivated by
this, our study delves into the utilization of the implicit knowledge embedded
within diffusion models to address challenges in cross-domain semantic
segmentation. This paper investigates the approach that leverages the sampling
and fusion techniques to harness the features of diffusion models efficiently.
Contrary to the simplistic migration applications characterized by prior
research, our finding reveals that the multi-step diffusion process inherent in
the diffusion model manifests more robust semantic features. We propose
DIffusion Feature Fusion (DIFF) as a backbone use for extracting and
integrating effective semantic representations through the diffusion process.
By leveraging the strength of text-to-image generation capability, we introduce
a new training framework designed to implicitly learn posterior knowledge from
it. Through rigorous evaluation in the contexts of domain generalization
semantic segmentation, we establish that our methodology surpasses preceding
approaches in mitigating discrepancies across distinct domains and attains the
state-of-the-art (SOTA) benchmark. Within the synthetic-to-real (syn-to-real)
context, our method significantly outperforms ResNet-based and
transformer-based backbone methods, achieving an average improvement of
$3.84\%$ mIoU across various datasets. The implementation code will be released
soon.

利用预训练扩散模型的内在知识，通过采样和融合技术，提出了一种新的扩散特征融合方法（DIFF）来提取和集成有效的语义表示，该方法在领域泛化语义分割和合成到真实情景中均优于先前的方法达到了最优性能。