Although diffusion models can generate high-quality human images, their
applications are limited by the instability in generating hands with correct
structures. Some previous works mitigate the problem by considering hand
structure yet struggle to maintain style consistency between refined malformed
hands and other image regions. In this paper, we aim to solve the problem of
inconsistency regarding hand structure and style. We propose a conditional
diffusion-based framework RHanDS to refine the hand region with the help of
decoupled structure and style guidance. Specifically, the structure guidance is
the hand mesh reconstructed from the malformed hand, serving to correct the
hand structure. The style guidance is a hand image, e.g., the malformed hand
itself, and is employed to furnish the style reference for hand refining. In
order to suppress the structure leakage when referencing hand style and
effectively utilize hand data to improve the capability of the model, we build
a multi-style hand dataset and introduce a twostage training strategy. In the
first stage, we use paired hand images for training to generate hands with the
same style as the reference. In the second stage, various hand images generated
based on the human mesh are used for training to enable the model to gain
control over the hand structure. We evaluate our method and counterparts on the
test dataset of the proposed multi-style hand dataset. The experimental results
show that RHanDS can effectively refine hands structure- and style- correctly
compared with previous methods. The codes and datasets will be available soon.

通过使用条件扩散框架来改善手部结构及风格一致性问题，本文提出了一种基于扩散的手部细化框架 RHanDS，利用解耦的结构和风格引导对手部区域进行优化，评估结果显示 RHanDS 可以有效地改善手部结构和风格问题。