The pose-guided person image generation task requires synthesizing
photorealistic images of humans in arbitrary poses. The existing approaches use
generative adversarial networks that do not necessarily maintain realistic
textures or need dense correspondences that struggle to handle complex
deformations and severe occlusions. In this work, we show how denoising
diffusion models can be applied for high-fidelity person image synthesis with
strong sample diversity and enhanced mode coverage of the learnt data
distribution. Our proposed Person Image Diffusion Model (PIDM) disintegrates
the complex transfer problem into a series of simpler forward-backward
denoising steps. This helps in learning plausible source-to-target
transformation trajectories that result in faithful textures and undistorted
appearance details. We introduce a 'texture diffusion module' based on
cross-attention to accurately model the correspondences between appearance and
pose information available in source and target images. Further, we propose
'disentangled classifier-free guidance' to ensure close resemblance between the
conditional inputs and the synthesized output in terms of both pose and
appearance information. Our extensive results on two large-scale benchmarks and
a user study demonstrate the photorealism of our proposed approach under
challenging scenarios. We also show how our generated images can help in
downstream tasks. Our code and models will be publicly released.

使用去噪扩散模型，我们提出了一种名为 PIDM 的人体图像扩散模型，解决了复杂的转换问题，并展示了在两个大型基准测试中的显着结果，以及如何在下游任务中使用生成的图像。

基于去噪扩散模型的人物图像合成

Person Image Synthesis via Denoising Diffusion Model

Pose-guided person image generation and animation aim to transform a source
person image to target poses. These tasks require spatial manipulation of
source data. However, Convolutional Neural Networks are limited by the lack of
ability to spatially transform the inputs. In this paper, we propose a
differentiable global-flow local-attention framework to reassemble the inputs
at the feature level. This framework first estimates global flow fields between
sources and targets. Then, corresponding local source feature patches are
sampled with content-aware local attention coefficients. We show that our
framework can spatially transform the inputs in an efficient manner. Meanwhile,
we further model the temporal consistency for the person image animation task
to generate coherent videos. The experiment results of both image generation
and animation tasks demonstrate the superiority of our model. Besides,
additional results of novel view synthesis and face image animation show that
our model is applicable to other tasks requiring spatial transformation. The
source code of our project is available at
this https URL

本文提出 “可微分的全局流本地注意力框架”，以空间转换输入，实现人像生成和动画等多项任务，同时通过建模时间一致性，生成协调的视频，该框架在多项任务中表现优异，包括新视角合成和面部图像动画。

姿态引导的人像生成与动画的深度空间转换

Deep Spatial Transformation for Pose-Guided Person Image Generation and  Animation

Pose-guided person image generation is to transform a source person image to
a target pose. This task requires spatial manipulations of source data.
However, Convolutional Neural Networks are limited by the lack of ability to
spatially transform the inputs. In this paper, we propose a differentiable
global-flow local-attention framework to reassemble the inputs at the feature
level. Specifically, our model first calculates the global correlations between
sources and targets to predict flow fields. Then, the flowed local patch pairs
are extracted from the feature maps to calculate the local attention
coefficients. Finally, we warp the source features using a content-aware
sampling method with the obtained local attention coefficients. The results of
both subjective and objective experiments demonstrate the superiority of our
model. Besides, additional results in video animation and view synthesis show
that our model is applicable to other tasks requiring spatial transformation.
Our source code is available at
this https URL

本文提出一种不同于传统 CNN 的可微分全局流本地注意力框架来进行姿势引导的人体图像生成，该模型通过预测流场来进行全局相应，通过提取特征图中的局部补丁对局部注意力系数进行计算来生成新的人体图像，实验结果表明该方法优于传统方法，同时也适用于其他要求空间变换的任务。