Diffusion model is a promising approach to image generation and has been
employed for Pose-Guided Person Image Synthesis (PGPIS) with competitive
performance. While existing methods simply align the person appearance to the
target pose, they are prone to overfitting due to the lack of a high-level
semantic understanding on the source person image. In this paper, we propose a
novel Coarse-to-Fine Latent Diffusion (CFLD) method for PGPIS. In the absence
of image-caption pairs and textual prompts, we develop a novel training
paradigm purely based on images to control the generation process of the
pre-trained text-to-image diffusion model. A perception-refined decoder is
designed to progressively refine a set of learnable queries and extract
semantic understanding of person images as a coarse-grained prompt. This allows
for the decoupling of fine-grained appearance and pose information controls at
different stages, and thus circumventing the potential overfitting problem. To
generate more realistic texture details, a hybrid-granularity attention module
is proposed to encode multi-scale fine-grained appearance features as bias
terms to augment the coarse-grained prompt. Both quantitative and qualitative
experimental results on the DeepFashion benchmark demonstrate the superiority
of our method over the state of the arts for PGPIS. Code is available at
this https URL

提出了一种新的粗粒度到细粒度潜在扩散方法，用于姿势引导的人物图像合成，并在 DeepFashion 基准上展示了其优越性。