We present in this work the first end-to-end deep learning based method that
predicts both 3D hand shape and pose from RGB images in the wild. Our network
consists of the concatenation of a deep convolutional encoder, and a fixed
model-based decoder. Given an input image, and optionally 2D joint detections
obtained from an independent CNN, the encoder predicts a set of hand and view
parameters. The decoder has two components: A pre-computed articulated mesh
deformation hand model that generates a 3D mesh from the hand parameters, and a
re-projection module controlled by the view parameters that projects the
generated hand into the image domain. We show that using the shape and pose
prior knowledge encoded in the hand model within a deep learning framework
yields state-of-the-art performance in 3D pose prediction from images on
standard benchmarks, and produces geometrically valid and plausible 3D
reconstructions. Additionally, we show that training with weak supervision in
the form of 2D joint annotations on datasets of images in the wild, in
conjunction with full supervision in the form of 3D joint annotations on
limited available datasets allows for good generalization to 3D shape and pose
predictions on images in the wild.

利用先前计算好的人工建模和先验知识，结合深度学习的方法实现了从 RGB 图像中预测手部形状和姿态，结果显示出在标准基准测试中表现出最新颖的 3D 姿态预测效果，同时在弱监督下训练的模型也能够很好的应用于在实际环境中的 3D 形状和姿态预测。