This work targets at using a general deep learning framework to synthesize
free-viewpoint images of arbitrary human performers, only requiring a sparse
number of camera views as inputs and skirting per-case fine-tuning. The large
variation of geometry and appearance, caused by articulated body poses, shapes
and clothing types, are the key bottlenecks of this task. To overcome these
challenges, we present a simple yet powerful framework, named Generalizable
Neural Performer (GNR), that learns a generalizable and robust neural body
representation over various geometry and appearance. Specifically, we compress
the light fields for novel view human rendering as conditional implicit neural
radiance fields from both geometry and appearance aspects. We first introduce
an Implicit Geometric Body Embedding strategy to enhance the robustness based
on both parametric 3D human body model and multi-view images hints. We further
propose a Screen-Space Occlusion-Aware Appearance Blending technique to
preserve the high-quality appearance, through interpolating source view
appearance to the radiance fields with a relax but approximate geometric
guidance.
To evaluate our method, we present our ongoing effort of constructing a
dataset with remarkable complexity and diversity. The dataset GeneBody-1.0,
includes over 360M frames of 370 subjects under multi-view cameras capturing,
performing a large variety of pose actions, along with diverse body shapes,
clothing, accessories and hairdos. Experiments on GeneBody-1.0 and ZJU-Mocap
show better robustness of our methods than recent state-of-the-art
generalizable methods among all cross-dataset, unseen subjects and unseen poses
settings. We also demonstrate the competitiveness of our model compared with
cutting-edge case-specific ones. Dataset, code and model will be made publicly
available.

使用深度学习技术，提出了一种名为 GNR 的通用神经演员框架，通过压缩轻场数据来从几何和外观方面生成新颖的视角下的人体影像，并在多视角摄像机下构建的数据集 GeneBody-1.0 上证明了其鲁棒性和通用性。