Hand gesture-to-gesture translation in the wild is a challenging task since
hand gestures can have arbitrary poses, sizes, locations and self-occlusions.
Therefore, this task requires a high-level understanding of the mapping between
the input source gesture and the output target gesture. To tackle this problem,
we propose a novel hand Gesture Generative Adversarial Network (GestureGAN).
GestureGAN consists of a single generator $G$ and a discriminator $D$, which
takes as input a conditional hand image and a target hand skeleton image.
GestureGAN utilizes the hand skeleton information explicitly, and learns the
gesture-to-gesture mapping through two novel losses, the color loss and the
cycle-consistency loss. The proposed color loss handles the issue of "channel
pollution" while back-propagating the gradients. In addition, we present the
Fr\'echet ResNet Distance (FRD) to evaluate the quality of generated images.
Extensive experiments on two widely used benchmark datasets demonstrate that
the proposed GestureGAN achieves state-of-the-art performance on the
unconstrained hand gesture-to-gesture translation task. Meanwhile, the
generated images are in high-quality and are photo-realistic, allowing them to
be used as data augmentation to improve the performance of a hand gesture
classifier. Our model and code are available at
this https URL

提出了一种新的手势生成对抗网络 (GestureGAN) ，通过显式地利用手部骨骼信息和两个新的 loss 函数：color loss 和 cycle-consistency loss 来实现手势翻译任务。在两个广泛使用的基准数据集上进行的大量实验表明，提出的 GestureGAN 在非约束性手势翻译任务中实现了最先进的性能，并生成高质量、逼真的图像，可用作改进手势分类器性能的数据增强。