Novel view synthesis (NVS) of multi-human scenes imposes challenges due to
the complex inter-human occlusions. Layered representations handle the
complexities by dividing the scene into multi-layered radiance fields, however,
they are mainly constrained to per-scene optimization making them inefficient.
Generalizable human view synthesis methods combine the pre-fitted 3D human
meshes with image features to reach generalization, yet they are mainly
designed to operate on single-human scenes. Another drawback is the reliance on
multi-step optimization techniques for parametric pre-fitting of the 3D body
models that suffer from misalignment with the images in sparse view settings
causing hallucinations in synthesized views. In this work, we propose,
GenLayNeRF, a generalizable layered scene representation for free-viewpoint
rendering of multiple human subjects which requires no per-scene optimization
and very sparse views as input. We divide the scene into multi-human layers
anchored by the 3D body meshes. We then ensure pixel-level alignment of the
body models with the input views through a novel end-to-end trainable module
that carries out iterative parametric correction coupled with multi-view
feature fusion to produce aligned 3D models. For NVS, we extract point-wise
image-aligned and human-anchored features which are correlated and fused using
self-attention and cross-attention modules. We augment low-level RGB values
into the features with an attention-based RGB fusion module. To evaluate our
approach, we construct two multi-human view synthesis datasets; DeepMultiSyn
and ZJU-MultiHuman. The results indicate that our proposed approach outperforms
generalizable and non-human per-scene NeRF methods while performing at par with
layered per-scene methods without test time optimization.

提出了一种适用于多人场景的通用分层场景表示方法，用于多人视点渲染，无需每场景优化且只需要很少的输入视图。通过将场景分割为多人分层光辐射场，并通过迭代参数修正和多视图特征融合实现与输入视图的像素级对齐的 3D 模型，进一步提出了新颖的端到端训练模块进行人体模型校正。在新颖视图综合和非人物优化场景 NeRF 方法上表现优越，而与分层场景方法在测试时间优化方面相当。