Reconstructing a 3D hand from a single-view RGB image is challenging due to
various hand configurations and depth ambiguity. To reliably reconstruct a 3D
hand from a monocular image, most state-of-the-art methods heavily rely on 3D
annotations at the training stage, but obtaining 3D annotations is expensive.
To alleviate reliance on labeled training data, we propose S2HAND, a
self-supervised 3D hand reconstruction network that can jointly estimate pose,
shape, texture, and the camera viewpoint. Specifically, we obtain geometric
cues from the input image through easily accessible 2D detected keypoints. To
learn an accurate hand reconstruction model from these noisy geometric cues, we
utilize the consistency between 2D and 3D representations and propose a set of
novel losses to rationalize outputs of the neural network. For the first time,
we demonstrate the feasibility of training an accurate 3D hand reconstruction
network without relying on manual annotations. Our experiments show that the
proposed method achieves comparable performance with recent fully-supervised
methods while using fewer supervision data.

本文提出了一种利用自我监督的三维手部重建网络，通过利用易于提取的二维关键点从输入图像中获取几何线索，并通过一套新颖的损失函数推理出网络输出，以进行精确的手部重建。与使用手动标注数据训练的方法相比，我们的方法减少了对手动标注的依赖，并展示了在更少的监督数据下达到可比较的性能。

基于模型的自监督学习实现的 3D 手部重建

Model-based 3D Hand Reconstruction via Self-Supervised Learning

We develop a system for modeling hand-object interactions in 3D from RGB
images that show a hand which is holding a novel object from a known category.
We design a Convolutional Neural Network (CNN) for Hand-held Object Pose and
Shape estimation called HOPS-Net and utilize prior work to estimate the hand
pose and configuration. We leverage the insight that information about the hand
facilitates object pose and shape estimation by incorporating the hand into
both training and inference of the object pose and shape as well as the
refinement of the estimated pose. The network is trained on a large synthetic
dataset of objects in interaction with a human hand. To bridge the gap between
real and synthetic images, we employ an image-to-image translation model
(Augmented CycleGAN) that generates realistically textured objects given a
synthetic rendering. This provides a scalable way of generating annotated data
for training HOPS-Net. Our quantitative experiments show that even noisy hand
parameters significantly help object pose and shape estimation. The qualitative
experiments show results of pose and shape estimation of objects held by a hand
"in the wild".

该研究提出了一种利用卷积神经网络和图像转化模型实现手持物体姿态和形状估计的方法，并使用合成数据训练模型，获得了在真实场景与合成场景数据上预测物体姿态和形状的良好结果。