In recent advancements in novel view synthesis, generalizable Neural Radiance
Fields (NeRF) based methods applied to human subjects have shown remarkable
results in generating novel views from few images. However, this generalization
ability cannot capture the underlying structural features of the skeleton
shared across all instances. Building upon this, we introduce HFNeRF: a novel
generalizable human feature NeRF aimed at generating human biomechanic features
using a pre-trained image encoder. While previous human NeRF methods have shown
promising results in the generation of photorealistic virtual avatars, such
methods lack underlying human structure or biomechanic features such as
skeleton or joint information that are crucial for downstream applications
including Augmented Reality (AR)/Virtual Reality (VR). HFNeRF leverages 2D
pre-trained foundation models toward learning human features in 3D using neural
rendering, and then volume rendering towards generating 2D feature maps. We
evaluate HFNeRF in the skeleton estimation task by predicting heatmaps as
features. The proposed method is fully differentiable, allowing to successfully
learn color, geometry, and human skeleton in a simultaneous manner. This paper
presents preliminary results of HFNeRF, illustrating its potential in
generating realistic virtual avatars with biomechanic features using NeRF.

HFNeRF 是一种基于 NeRF 的新型可推广的人体特征生成方法，通过预训练的图像编码器学习人体特征，结合神经渲染和体素渲染生成 2D 特征地图。该方法可以同时学习颜色、几何和人体骨骼，在生成逼真的具有生物力学特征的虚拟人物方面具有潜力。