We tackle the challenging task of estimating global 3D joint locations for
both hands via only monocular RGB input images. We propose a novel multi-stage
convolutional neural network based pipeline that accurately segments and
locates the hands despite occlusion between two hands and complex background
noise and estimates the 2D and 3D canonical joint locations without any depth
information. Global joint locations with respect to the camera origin are
computed using the hand pose estimations and the actual length of the key bone
with a novel projection algorithm. To train the CNNs for this new task, we
introduce a large-scale synthetic 3D hand pose dataset. We demonstrate that our
system outperforms previous works on 3D canonical hand pose estimation
benchmark datasets with RGB-only information. Additionally, we present the
first work that achieves accurate global 3D hand tracking on both hands using
RGB-only inputs and provide extensive quantitative and qualitative evaluation.

本文提出了一种基于卷积神经网络的多阶段流水线方法，仅通过单眼红绿蓝彩色图像在众多的干扰因素下，准确分割和定位手部区域并估计 2D 和 3D 的关节点位置；采用新颖的投影算法计算相对相机坐标系的全局关节点位置，为此引入大规模合成 3D 手部姿态数据集。在 RGB-only 信息下，本文超越了以往的 3D 典型手部姿态估计基准数据集；此外，本文提供了 RGB-only 输入下首个实现双手的准确全局 3D 手部跟踪，并进行了广泛的定量和定性评估。