Semantic image synthesis aims to generate high-quality images given semantic
conditions, i.e. segmentation masks and style reference images. Existing
methods widely adopt generative adversarial networks (GANs). GANs take all
conditional inputs and directly synthesize images in a single forward step. In
this paper, semantic image synthesis is treated as an image denoising task and
is handled with a novel image-to-image diffusion model (IIDM). Specifically,
the style reference is first contaminated with random noise and then
progressively denoised by IIDM, guided by segmentation masks. Moreover, three
techniques, refinement, color-transfer and model ensembles, are proposed to
further boost the generation quality. They are plug-in inference modules and do
not require additional training. Extensive experiments show that our IIDM
outperforms existing state-of-the-art methods by clear margins. Further
analysis is provided via detailed demonstrations. We have implemented IIDM
based on the Jittor framework; code is available at
this https URL

本文提出了一种基于图像扩散模型的语义图像合成方法，通过引入随机噪声和逐渐去噪的方式，以分割掩模为指导生成高质量图像，并进一步通过细化、颜色转移和模型集成等技术提高合成质量。大量实验证明本方法在性能上明显超越了现有的最先进方法。