Reconstructing hand-held objects from monocular RGB images is an appealing
yet challenging task. In this task, contacts between hands and objects provide
important cues for recovering the 3D geometry of the hand-held objects. Though
recent works have employed implicit functions to achieve impressive progress,
they ignore formulating contacts in their frameworks, which results in
producing less realistic object meshes. In this work, we explore how to model
contacts in an explicit way to benefit the implicit reconstruction of hand-held
objects. Our method consists of two components: explicit contact prediction and
implicit shape reconstruction. In the first part, we propose a new subtask of
directly estimating 3D hand-object contacts from a single image. The part-level
and vertex-level graph-based transformers are cascaded and jointly learned in a
coarse-to-fine manner for more accurate contact probabilities. In the second
part, we introduce a novel method to diffuse estimated contact states from the
hand mesh surface to nearby 3D space and leverage diffused contact
probabilities to construct the implicit neural representation for the
manipulated object. Benefiting from estimating the interaction patterns between
the hand and the object, our method can reconstruct more realistic object
meshes, especially for object parts that are in contact with hands. Extensive
experiments on challenging benchmarks show that the proposed method outperforms
the current state of the arts by a great margin.

本文提出了一种新的方法，通过建模手部与物体之间的互动模式，达到更准确、更真实的物体重建，包括手部接触物体的部分。该方法由两部分组成：显式接触预测和隐式形状重建，并在挑战性数据集上表现出比当前艺术水平更出色的性能。