We tackle the challenging task of estimating global 3D joint locations for
both hands via only monocular RGB input images. We propose a novel multi-stage
convolutional neural network based pipeline that accurately segments and
locates the hands despite occlusion between two hands and complex background
noise and estimates the 2D and 3D canonical joint locations without any depth
information. Global joint locations with respect to the camera origin are
computed using the hand pose estimations and the actual length of the key bone
with a novel projection algorithm. To train the CNNs for this new task, we
introduce a large-scale synthetic 3D hand pose dataset. We demonstrate that our
system outperforms previous works on 3D canonical hand pose estimation
benchmark datasets with RGB-only information. Additionally, we present the
first work that achieves accurate global 3D hand tracking on both hands using
RGB-only inputs and provide extensive quantitative and qualitative evaluation.

本文提出了一种基于卷积神经网络的多阶段流水线方法，仅通过单眼红绿蓝彩色图像在众多的干扰因素下，准确分割和定位手部区域并估计 2D 和 3D 的关节点位置；采用新颖的投影算法计算相对相机坐标系的全局关节点位置，为此引入大规模合成 3D 手部姿态数据集。在 RGB-only 信息下，本文超越了以往的 3D 典型手部姿态估计基准数据集；此外，本文提供了 RGB-only 输入下首个实现双手的准确全局 3D 手部跟踪，并进行了广泛的定量和定性评估。

使用单目 RGB 进行双手全局 3D 姿态估计

Two-hand Global 3D Pose Estimation Using Monocular RGB

In this paper, we present a HAnd Mesh Recovery (HAMR) framework to tackle the
problem of reconstructing the full 3D mesh of a human hand from a single RGB
image. In contrast to existing research on 2D or 3D hand pose estimation from
RGB or/and depth image data, HAMR can provide a more expressive and useful mesh
representation for monocular hand image understanding. In particular, the mesh
representation is achieved by parameterizing a generic 3D hand model with shape
and relative 3D joint angles. By utilizing this mesh representation, we can
easily compute the 3D joint locations via linear interpolations between the
vertexes of the mesh, while obtain the 2D joint locations with a projection of
the 3D joints.To this end, a differentiable re-projection loss can be defined
in terms of the derived representations and the ground-truth labels, thus
making our framework end-to-end trainable.Qualitative experiments show that our
framework is capable of recovering appealing 3D hand mesh even in the presence
of severe occlusions.Quantitatively, our approach also outperforms the
state-of-the-art methods for both 2D and 3D hand pose estimation from a
monocular RGB image on several benchmark datasets.

本文提出了 HAnd Mesh Recovery (HAMR) 框架，通过参数化通用 3D 手模型的形状和相对 3D 关节角度，提供了更多表达和更有用的网格表示，该框架可从单个 RGB 图像重建人手的完整 3D 网格，并具有较好的表现。

从单目 RGB 图像恢复端到端的手部网格

End-to-end Hand Mesh Recovery from a Monocular RGB Image

While many recent hand pose estimation methods critically rely on a training
set of labelled frames, the creation of such a dataset is a challenging task
that has been overlooked so far. As a result, existing datasets are limited to
a few sequences and individuals, with limited accuracy, and this prevents these
methods from delivering their full potential. We propose a semi-automated
method for efficiently and accurately labeling each frame of a hand depth video
with the corresponding 3D locations of the joints: The user is asked to provide
only an estimate of the 2D reprojections of the visible joints in some
reference frames, which are automatically selected to minimize the labeling
work by efficiently optimizing a sub-modular loss function. We then exploit
spatial, temporal, and appearance constraints to retrieve the full 3D poses of
the hand over the complete sequence. We show that this data can be used to
train a recent state-of-the-art hand pose estimation method, leading to
increased accuracy. The code and dataset can be found on our website
this https URL

本文提出了一种半自动标记方法，该方法可以高效地标记手深度视频的每一帧，从而获得 3D 关节位置数据用于训练手势估计模型，并在手势识别领域取得了最先进的精度。

高效创建三维训练数据以进行手部姿态细化估计

Efficiently Creating 3D Training Data for Fine Hand Pose Estimation

We introduce and evaluate several architectures for Convolutional Neural
Networks to predict the 3D joint locations of a hand given a depth map. We
first show that a prior on the 3D pose can be easily introduced and
significantly improves the accuracy and reliability of the predictions. We also
show how to use context efficiently to deal with ambiguities between fingers.
These two contributions allow us to significantly outperform the
state-of-the-art on several challenging benchmarks, both in terms of accuracy
and computation times.

本文研究使用卷积神经网络结构，预测手的三维关节位置，并通过引入关节先验和有效利用上下文来提高方案的预测精确性和可靠性，在多个基准测试中都表现出了远胜于现有技术的高精度和较快的计算时间。