Despite recent advances in 3D pose estimation of human hands, especially
thanks to the advent of CNNs and depth cameras, this task is still far from
being solved. This is mainly due to the highly non-linear dynamics of fingers,
which make hand model training a challenging task. In this paper, we exploit a
novel hierarchical tree-like structured CNN, in which branches are trained to
become specialized in predefined subsets of hand joints, called local poses. We
further fuse local pose features, extracted from hierarchical CNN branches, to
learn higher order dependencies among joints in the final pose by end-to-end
training. Lastly, the loss function used is also defined to incorporate
appearance and physical constraints about doable hand motion and deformation.
Finally, we introduce a non-rigid data augmentation approach to increase the
amount of training depth data. Experimental results suggest that feeding a
tree-shaped CNN, specialized in local poses, into a fusion network for modeling
joints correlations and dependencies, helps to increase the precision of final
estimations, outperforming state-of-the-art results on NYU and SyntheticHand
datasets.

本文提出了一种基于层次结构的 CNN，通过对手的手指等局部姿势进行预训练，并将这些特征融合起来，以学习最终姿态中的关节之间的更高级别的依赖关系，同时，在损失函数中加入与手的运动和变形相关的外观和物理约束，并引入一种非刚性数据扩充方法，实验结果表明，基于局部姿态的树形 CNN 模型结合关节相关性和依赖性的融合网络，能够提高最终姿态估计的精度，优于现有的 NYU 和 SyntheticHand 数据集结果。