Recent years have seen significant progress in human image generation, particularly with the advancements in diffusion models. However, existing diffusion methods encounter challenges when producing consistent hand anatomy and the generated images often lack precise control over the hand pose. To address this limitation, we introduce a novel approach to pose-conditioned human image generation, dividing the process into two stages: hand generation and subsequent body out-painting around the hands. We propose training the hand generator in a multi-task setting to produce both hand images and their corresponding segmentation masks, and employ the trained model in the first stage of generation. An adapted ControlNet model is then used in the second stage to outpaint the body around the generated hands, producing the final result. A novel blending technique is introduced to preserve the hand details during the second stage that combines the results of both stages in a coherent way. This involves sequential expansion of the out-painted region while fusing the latent representations, to ensure a seamless and cohesive synthesis of the final image. Experimental evaluations demonstrate the superiority of our proposed method over state-of-the-art techniques, in both pose accuracy and image quality, as validated on the HaGRID dataset. Our approach not only enhances the quality of the generated hands but also offers improved control over hand pose, advancing the capabilities of pose-conditioned human image generation. The source code of the proposed approach is available at https://github.com/apelykh/hand-to-diffusion.

我们引入了一种新颖的姿势条件的人体图像生成方法，将生成过程分为两个阶段：手的生成和在手周围绘制身体。我们将手生成器训练在多任务设置中，以产生手图像及其相应的分割掩模，并将训练后的模型用于生成的第一阶段。然后，在第二阶段中使用改进的ControlNet模型来绘制生成的手周围的身体，产生最终的结果。我们引入了一种新颖的混合技术，在第二阶段中保留手部细节，以一种连贯的方式结合两个阶段的结果。实验评估证明了我们提出的方法在姿势准确性和图像质量上优于现有技术，在HaGRID数据集上得到验证。我们的方法不仅提高了生成手部的质量，还提供了对手势的改进控制，推进了姿势条件的人体图像生成的能力。

帮助扩散模型：改进条件人类图像生成的两阶段方法