Diffusion model is a promising approach to image generation and has been
employed for Pose-Guided Person Image Synthesis (PGPIS) with competitive
performance. While existing methods simply align the person appearance to the
target pose, they are prone to overfitting due to the lack of a high-level
semantic understanding on the source person image. In this paper, we propose a
novel Coarse-to-Fine Latent Diffusion (CFLD) method for PGPIS. In the absence
of image-caption pairs and textual prompts, we develop a novel training
paradigm purely based on images to control the generation process of the
pre-trained text-to-image diffusion model. A perception-refined decoder is
designed to progressively refine a set of learnable queries and extract
semantic understanding of person images as a coarse-grained prompt. This allows
for the decoupling of fine-grained appearance and pose information controls at
different stages, and thus circumventing the potential overfitting problem. To
generate more realistic texture details, a hybrid-granularity attention module
is proposed to encode multi-scale fine-grained appearance features as bias
terms to augment the coarse-grained prompt. Both quantitative and qualitative
experimental results on the DeepFashion benchmark demonstrate the superiority
of our method over the state of the arts for PGPIS. Code is available at
this https URL

提出了一种新的粗粒度到细粒度潜在扩散方法，用于姿势引导的人物图像合成，并在 DeepFashion 基准上展示了其优越性。

姿势引导的人体图像合成的粗细潜扩散

Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis

Pose-guided person image synthesis task requires re-rendering a reference
image, which should have a photorealistic appearance and flawless pose
transfer. Since person images are highly structured, existing approaches
require dense connections for complex deformations and occlusions because these
are generally handled through multi-level warping and masking in latent space.
But the feature maps generated by convolutional neural networks do not have
equivariance, and hence even the multi-level warping does not have a perfect
pose alignment. Inspired by the ability of the diffusion model to generate
photorealistic images from the given conditional guidance, we propose recurrent
pose alignment to provide pose-aligned texture features as conditional
guidance. Moreover, we propose gradient guidance from pose interaction fields,
which output the distance from the valid pose manifold given a target pose as
input. This helps in learning plausible pose transfer trajectories that result
in photorealism and undistorted texture details. Extensive results on two
large-scale benchmarks and a user study demonstrate the ability of our proposed
approach to generate photorealistic pose transfer under challenging scenarios.
Additionally, we prove the efficiency of gradient guidance in pose-guided image
generation on the HumanArt dataset with fine-tuned stable diffusion.

本研究提出了一种基于循环姿态对齐和梯度引导的方法，可以生成具有真实外观和无瑕疵姿态转移的人物图像。经过广泛的实验证明，该方法能够在复杂场景下生成具有真实感的姿态转移，并且通过人工评测证明了其有效性。

RePoseDM：用于姿势导向图像合成的循环姿势对齐和梯度引导

RePoseDM: Recurrent Pose Alignment and Gradient Guidance for Pose Guided  Image Synthesis

Recent work has showcased the significant potential of diffusion models in
pose-guided person image synthesis. However, owing to the inconsistency in pose
between the source and target images, synthesizing an image with a distinct
pose, relying exclusively on the source image and target pose information,
remains a formidable challenge. This paper presents Progressive Conditional
Diffusion Models (PCDMs) that incrementally bridge the gap between person
images under the target and source poses through three stages. Specifically, in
the first stage, we design a simple prior conditional diffusion model that
predicts the global features of the target image by mining the global alignment
relationship between pose coordinates and image appearance. Then, the second
stage establishes a dense correspondence between the source and target images
using the global features from the previous stage, and an inpainting
conditional diffusion model is proposed to further align and enhance the
contextual features, generating a coarse-grained person image. In the third
stage, we propose a refining conditional diffusion model to utilize the
coarsely generated image from the previous stage as a condition, achieving
texture restoration and enhancing fine-detail consistency. The three-stage
PCDMs work progressively to generate the final high-quality and high-fidelity
synthesized image. Both qualitative and quantitative results demonstrate the
consistency and photorealism of our proposed PCDMs under challenging
scenarios.The code and model will be available at
this https URL

该研究论文提出了一种渐进条件扩散模型（PCDMs），通过三个阶段逐渐弥合目标姿势和源姿势下的人像之间的差距，并生成高质量、高保真的综合图像。