We propose a novel framework to reconstruct accurate appearance and geometry
with neural radiance fields (NeRF) for interacting hands, enabling the
rendering of photo-realistic images and videos for gesture animation from
arbitrary views. Given multi-view images of a single hand or interacting hands,
an off-the-shelf skeleton estimator is first employed to parameterize the hand
poses. Then we design a pose-driven deformation field to establish
correspondence from those different poses to a shared canonical space, where a
pose-disentangled NeRF for one hand is optimized. Such unified modeling
efficiently complements the geometry and texture cues in rarely-observed areas
for both hands. Meanwhile, we further leverage the pose priors to generate
pseudo depth maps as guidance for occlusion-aware density learning. Moreover, a
neural feature distillation method is proposed to achieve cross-domain
alignment for color optimization. We conduct extensive experiments to verify
the merits of our proposed HandNeRF and report a series of state-of-the-art
results both qualitatively and quantitatively on the large-scale InterHand2.6M
dataset.

该文提出了一个新的框架，使用神经辐射场（NeRF）重建手的外观和几何，使得能够从任意视角呈现手势动画的逼真图像和视频，并利用姿势驱动的变形场等方法实现了手外观与几何的合理映射，同时在密度学习中使用了伪深度图进行遮挡处理，最终在多项实验证明了该方法的有效性。