Existing methods proposed for hand reconstruction tasks usually parameterize
a generic 3D hand model or predict hand mesh positions directly. The parametric
representations consisting of hand shapes and rotational poses are more stable,
while the non-parametric methods can predict more accurate mesh positions. In
this paper, we propose to reconstruct meshes and estimate MANO parameters of
two hands from a single RGB image simultaneously to utilize the merits of two
kinds of hand representations. To fulfill this target, we propose novel
Mesh-Mano interaction blocks (MMIBs), which take mesh vertices positions and
MANO parameters as two kinds of query tokens. MMIB consists of one graph
residual block to aggregate local information and two transformer encoders to
model long-range dependencies. The transformer encoders are equipped with
different asymmetric attention masks to model the intra-hand and inter-hand
attention, respectively. Moreover, we introduce the mesh alignment refinement
module to further enhance the mesh-image alignment. Extensive experiments on
the InterHand2.6M benchmark demonstrate promising results over the
state-of-the-art hand reconstruction methods.

本文提出了一种利用素材学习获取双手的网络来实现手的重建，其中考虑了两种不同的手表示法。通过引入 Mesh-Mano interaction blocks（MMIBs）以及 mesh alignment refinement module，我们的方法在 InterHand2.6M 基准上的表现优于现有的手重建方法，生成效果明显。