Modeling hand-object interactions is a fundamentally challenging task in 3D
computer vision. Despite remarkable progress that has been achieved in this
field, existing methods still fail to synthesize the hand-object interaction
photo-realistically, suffering from degraded rendering quality caused by the
heavy mutual occlusions between the hand and the object, and inaccurate
hand-object pose estimation. To tackle these challenges, we present a novel
free-viewpoint rendering framework, Neural Contact Radiance Field (NCRF), to
reconstruct hand-object interactions from a sparse set of videos. In
particular, the proposed NCRF framework consists of two key components: (a) A
contact optimization field that predicts an accurate contact field from 3D
query points for achieving desirable contact between the hand and the object.
(b) A hand-object neural radiance field to learn an implicit hand-object
representation in a static canonical space, in concert with the specifically
designed hand-object motion field to produce observation-to-canonical
correspondences. We jointly learn these key components where they mutually help
and regularize each other with visual and geometric constraints, producing a
high-quality hand-object reconstruction that achieves photo-realistic novel
view synthesis. Extensive experiments on HO3D and DexYCB datasets show that our
approach outperforms the current state-of-the-art in terms of both rendering
quality and pose estimation accuracy.

手 - 物体交互模型是三维计算机视觉中一项根本性的挑战任务。尽管在这一领域已取得了显著的进展，但现有方法仍无法实现逼真的手 - 物体交互合成，因为手和物体之间存在严重的互相遮挡和不准确的姿态估计所导致的渲染质量下降。为了解决这些挑战，我们提出了一种新颖的自由视点渲染框架，即神经接触辐射场（NCRF），用于从稀疏的视频集合中重建手 - 物体交互。尤其值得注意的是，我们的 NCRF 框架由两个关键组件组成：（a）接触优化场，用于从 3D 查询点预测准确的接触场，以实现手和物体之间的理想接触；（b）手 - 物体神经辐射场，用于在静态规范空间中学习隐式的手 - 物体表示，与专门设计的手 - 物体运动场结合以产生观察到规范的对应关系。我们共同学习这些关键组件，它们相互帮助并相互规范，通过视觉和几何约束生成高质量的手 - 物体重建，实现逼真的新视图合成。在 HO3D 和 DexYCB 数据集上进行了大量实验证明我们的方法在渲染质量和姿态估计准确性方面优于当前最先进的方法。