Our work aims to reconstruct a 3D object that is held and rotated by a hand
in front of a static RGB camera. Previous methods that use implicit neural
representations to recover the geometry of a generic hand-held object from
multi-view images achieved compelling results in the visible part of the
object. However, these methods falter in accurately capturing the shape within
the hand-object contact region due to occlusion. In this paper, we propose a
novel method that deals with surface reconstruction under occlusion by
incorporating priors of 2D occlusion elucidation and physical contact
constraints. For the former, we introduce an object amodal completion network
to infer the 2D complete mask of objects under occlusion. To ensure the
accuracy and view consistency of the predicted 2D amodal masks, we devise a
joint optimization method for both amodal mask refinement and 3D
reconstruction. For the latter, we impose penetration and attraction
constraints on the local geometry in contact regions. We evaluate our approach
on HO3D and HOD datasets and demonstrate that it outperforms the
state-of-the-art methods in terms of reconstruction surface quality, with an
improvement of $52\%$ on HO3D and $20\%$ on HOD. Project webpage:
this https URL

本文提出一种新方法，通过引入 2D 遮挡澄清和物理接触约束，从而处理遮挡下的表面重建问题，该方法在测试集上表现优于现有方法，HO3D 效果提高了 52%，HOD 效果提高了 20%。