3D interacting hand reconstruction is essential to facilitate human-machine
interaction and human behaviors understanding. Previous works in this field
either rely on auxiliary inputs such as depth images or they can only handle a
single hand if monocular single RGB images are used. Single-hand methods tend
to generate collided hand meshes, when applied to closely interacting hands,
since they cannot model the interactions between two hands explicitly. In this
paper, we make the first attempt to reconstruct 3D interacting hands from
monocular single RGB images. Our method can generate 3D hand meshes with both
precise 3D poses and minimal collisions. This is made possible via a two-stage
framework. Specifically, the first stage adopts a convolutional neural network
to generate coarse predictions that tolerate collisions but encourage
pose-accurate hand meshes. The second stage progressively ameliorates the
collisions through a series of factorized refinements while retaining the
preciseness of 3D poses. We carefully investigate potential implementations for
the factorized refinement, considering the trade-off between efficiency and
accuracy. Extensive quantitative and qualitative results on large-scale
datasets such as InterHand2.6M demonstrate the effectiveness of the proposed
approach.

本文提出了一种通过卷积神经网络和分步优化的方式从单个 RGB 图像中重建精确的三维手部网格模型，以促进人机交互和人类行为理解，并在 InterHand2.6M 等大规模数据集上进行了广泛的定量和定性分析。

通过考虑碰撞感知的因式化优化，实现交互手的单目三维重建

Monocular 3D Reconstruction of Interacting Hands via Collision-Aware  Factorized Refinements

We propose a robust and accurate method for estimating the 3D poses of two
hands in close interaction from a single color image. This is a very
challenging problem, as large occlusions and many confusions between the joints
may happen. State-of-the-art methods solve this problem by regressing a heatmap
for each joint, which requires solving two problems simultaneously: localizing
the joints and recognizing them. In this work, we propose to separate these
tasks by relying on a CNN to first localize joints as 2D keypoints, and on
self-attention between the CNN features at these keypoints to associate them
with the corresponding hand joint. The resulting architecture, which we call
"Keypoint Transformer", is highly efficient as it achieves state-of-the-art
performance with roughly half the number of model parameters on the
InterHand2.6M dataset. We also show it can be easily extended to estimate the
3D pose of an object manipulated by one or two hands with high performance.
Moreover, we created a new dataset of more than 75,000 images of two hands
manipulating an object fully annotated in 3D and will make it publicly
available.

本文提出了一种名为 “Keypoint Transformer” 的深度神经网络结构，该结构可以通过二维关键点自注意力机制提取手部三维位姿，实验结果表明该算法在 InterHand2.6M 数据集上性能优于当前所有算法，并可以扩展到双手操纵物体的三维位姿估计，同时作者还提供了超过 75,000 个手部交互的三维场景数据集。

Keypoint Transformer：用于精确 3D 姿态估计中解决具有挑战性的手部和物体交互中的关键点标识问题

Keypoint Transformer: Solving Joint Identification in Challenging Hands  and Object Interactions for Accurate 3D Pose Estimation

Analysis of hand-hand interactions is a crucial step towards better
understanding human behavior. However, most researches in 3D hand pose
estimation have focused on the isolated single hand case. Therefore, we firstly
propose (1) a large-scale dataset, InterHand2.6M, and (2) a baseline network,
InterNet, for 3D interacting hand pose estimation from a single RGB image. The
proposed InterHand2.6M consists of \textbf{2.6M labeled single and interacting
hand frames} under various poses from multiple subjects. Our InterNet
simultaneously performs 3D single and interacting hand pose estimation. In our
experiments, we demonstrate big gains in 3D interacting hand pose estimation
accuracy when leveraging the interacting hand data in InterHand2.6M. We also
report the accuracy of InterNet on InterHand2.6M, which serves as a strong
baseline for this new dataset. Finally, we show 3D interacting hand pose
estimation results from general images. Our code and dataset are available at
this https URL

该篇论文针对手 - 手互动的行为研究提出了 InterNet 网络和 InterHand2.6M 数据库，用于从单个 RGB 图像中实现三维互动手势姿态估计，获得更准确的结果。