We present a method for recovering the dense 3D surface of the hand by
regressing the vertex coordinates of a mesh model from a single depth map. To
this end, we use a two-stage 2D fully convolutional network architecture. In
the first stage, the network estimates a dense correspondence field for every
pixel on the depth map or image grid to the mesh grid. In the second stage, we
design a differentiable operator to map features learned from the previous
stage and regress a 3D coordinate map on the mesh grid. Finally, we sample from
the mesh grid to recover the mesh vertices, and fit it an articulated template
mesh in closed form. During inference, the network can predict all the mesh
vertices, transformation matrices for every joint and the joint coordinates in
a single forward pass. When given supervision on the sparse key-point
coordinates, our method achieves state-of-the-art accuracy on NYU dataset for
key point localization while recovering mesh vertices and a dense
correspondence map. Our framework can also be learned through self-supervision
by minimizing a set of data fitting and kinematic prior terms. With
multi-camera rig during training to resolve self-occlusion, it can perform
competitively with strongly supervised methods Without any human annotation.

本文提出了一种通过从单个深度图中回归网格模型顶点坐标来恢复手部密集 3D 表面的方法，使用两阶段 2D 完全卷积网络架构和可微分算子来实现，能够通过自我监督实现更好的效果，表现优异。

从单一深度图中回归手部网格顶点的双网格网络

Dual Grid Net: hand mesh vertex regression from single depth maps

We propose an end-to-end deep learning architecture that produces a 3D shape
in triangular mesh from a single color image. Limited by the nature of deep
neural network, previous methods usually represent a 3D shape in volume or
point cloud, and it is non-trivial to convert them to the more ready-to-use
mesh model. Unlike the existing methods, our network represents 3D mesh in a
graph-based convolutional neural network and produces correct geometry by
progressively deforming an ellipsoid, leveraging perceptual features extracted
from the input image. We adopt a coarse-to-fine strategy to make the whole
deformation procedure stable, and define various of mesh related losses to
capture properties of different levels to guarantee visually appealing and
physically accurate 3D geometry. Extensive experiments show that our method not
only qualitatively produces mesh model with better details, but also achieves
higher 3D shape estimation accuracy compared to the state-of-the-art.

本文提出了一种基于图卷积神经网络的端到端深度学习架构，从单个彩色图像中生成三角网格的 3D 形状，并采用粗到细的策略、不同级别的网格相关损失来保证生成的几何形状具有良好的视觉效果和物理精度。