We present a method that enables synthesizing novel views and novel poses of
arbitrary human performers from sparse multi-view images. A key ingredient of
our method is a hybrid appearance blending module that combines the advantages
of the implicit body NeRF representation and image-based rendering. Existing
generalizable human NeRF methods that are conditioned on the body model have
shown robustness against the geometric variation of arbitrary human performers.
Yet they often exhibit blurry results when generalized onto unseen identities.
Meanwhile, image-based rendering shows high-quality results when sufficient
observations are available, whereas it suffers artifacts in sparse-view
settings. We propose Neural Image-based Avatars (NIA) that exploits the best of
those two methods: to maintain robustness under new articulations and
self-occlusions while directly leveraging the available (sparse) source view
colors to preserve appearance details of new subject identities. Our hybrid
design outperforms recent methods on both in-domain identity generalization as
well as challenging cross-dataset generalization settings. Also, in terms of
the pose generalization, our method outperforms even the per-subject optimized
animatable NeRF methods. The video results are available at
this https URL

本文提出一种基于稀疏多视图图像的神经图像化头像生成方法，利用了近年来提出的体视图下的神经反射场（NeRF）模型及图像合成技术，提高了新姿势下头像的变形能力和形体形变下姿势生成的只优。

神经图像化人头像：适用于人类化身建模的通用辐射场

Neural Image-based Avatars: Generalizable Radiance Fields for Human  Avatar Modeling

We present a novel neural implicit representation for articulated human
bodies. Compared to explicit template meshes, neural implicit body
representations provide an efficient mechanism for modeling interactions with
the environment, which is essential for human motion reconstruction and
synthesis in 3D scenes. However, existing neural implicit bodies suffer from
either poor generalization on highly articulated poses or slow inference time.
In this work, we observe that prior knowledge about the human body's shape and
kinematic structure can be leveraged to improve generalization and efficiency.
We decompose the full-body geometry into local body parts and employ a
part-aware encoder-decoder architecture to learn neural articulated occupancy
that models complex deformations locally. Our local shape encoder represents
the body deformation of not only the corresponding body part but also the
neighboring body parts. The decoder incorporates the geometric constraints of
local body shape which significantly improves pose generalization. We
demonstrate that our model is suitable for resolving self-intersections and
collisions with 3D environments. Quantitative and qualitative experiments show
that our method largely outperforms existing solutions in terms of both
efficiency and accuracy. The code and models are available at
this https URL

本研究提出了一种新型的神经隐式表示方法，旨在有效地对人体的关节运动进行建模、重构和合成，该方法利用具有局部感知能力的编码器 - 解码器结构将全身几何形状分解成局部身体部位，并学习局部复杂变形的神经隐式占据，同时考虑到身体部位的几何限制来提高姿态泛化能力，在 3D 环境中易于处理自相交和碰撞，实验结果表明，该模型在效率和准确性方面明显优于现有的解决方案。