Neural radiance fields (NeRFs) are promising 3D representations for scenes,
objects, and humans. However, most existing methods require multi-view inputs
and per-scene training, which limits their real-life applications. Moreover,
current methods focus on single-subject cases, leaving scenes of interacting
hands that involve severe inter-hand occlusions and challenging view variations
remain unsolved. To tackle these issues, this paper proposes a generalizable
visibility-aware NeRF (VA-NeRF) framework for interacting hands. Specifically,
given an image of interacting hands as input, our VA-NeRF first obtains a
mesh-based representation of hands and extracts their corresponding geometric
and textural features. Subsequently, a feature fusion module that exploits the
visibility of query points and mesh vertices is introduced to adaptively merge
features of both hands, enabling the recovery of features in unseen areas.
Additionally, our VA-NeRF is optimized together with a novel discriminator
within an adversarial learning paradigm. In contrast to conventional
discriminators that predict a single real/fake label for the synthesized image,
the proposed discriminator generates a pixel-wise visibility map, providing
fine-grained supervision for unseen areas and encouraging the VA-NeRF to
improve the visual quality of synthesized images. Experiments on the
Interhand2.6M dataset demonstrate that our proposed VA-NeRF outperforms
conventional NeRFs significantly. Project Page:
https://github.com/XuanHuang0/VANeRF.

这篇论文提出了一种可推广的可见性感知交互手部 NeRF（VA-NeRF）框架，通过利用可见性信息和改进的对抗学习，能够更好地还原具有交互的手部场景中不可见区域的特征，从而在 Interhand2.6M 数据集上取得显著的性能提升。