Predicting natural and diverse 3D hand gestures from the upper body dynamics
is a practical yet challenging task in virtual avatar creation. Previous works
usually overlook the asymmetric motions between two hands and generate two
hands in a holistic manner, leading to unnatural results. In this work, we
introduce a novel bilateral hand disentanglement based two-stage 3D hand
generation method to achieve natural and diverse 3D hand prediction from body
dynamics. In the first stage, we intend to generate natural hand gestures by
two hand-disentanglement branches. Considering the asymmetric gestures and
motions of two hands, we introduce a Spatial-Residual Memory (SRM) module to
model spatial interaction between the body and each hand by residual learning.
To enhance the coordination of two hand motions wrt. body dynamics
holistically, we then present a Temporal-Motion Memory (TMM) module. TMM can
effectively model the temporal association between body dynamics and two hand
motions. The second stage is built upon the insight that 3D hand predictions
should be non-deterministic given the sequential body postures. Thus, we
further diversify our 3D hand predictions based on the initial output from the
stage one. Concretely, we propose a Prototypical-Memory Sampling Strategy (PSS)
to generate the non-deterministic hand gestures by gradient-based Markov Chain
Monte Carlo (MCMC) sampling. Extensive experiments demonstrate that our method
outperforms the state-of-the-art models on the B2H dataset and our newly
collected TED Hands dataset. The dataset and code are available at
this https URL

本文介绍一种新型双手解缠的两阶段三维手势生成方法，以从身体动态中实现自然且多样化的三维手势预测，并通过渐进式采样进一步多样化手势预测。