Creating a digital human avatar that is relightable, drivable, and
photorealistic is a challenging and important problem in Vision and Graphics.
Humans are highly articulated creating pose-dependent appearance effects like
self-shadows and wrinkles, and skin as well as clothing require complex and
space-varying BRDF models. While recent human relighting approaches can recover
plausible material-light decompositions from multi-view video, they do not
generalize to novel poses and still suffer from visual artifacts. To address
this, we propose Relightable Neural Actor, the first video-based method for
learning a photorealistic neural human model that can be relighted, allows
appearance editing, and can be controlled by arbitrary skeletal poses.
Importantly, for learning our human avatar, we solely require a multi-view
recording of the human under a known, but static lighting condition. To achieve
this, we represent the geometry of the actor with a drivable density field that
models pose-dependent clothing deformations and provides a mapping between 3D
and UV space, where normal, visibility, and materials are encoded. To evaluate
our approach in real-world scenarios, we collect a new dataset with four actors
recorded under different light conditions, indoors and outdoors, providing the
first benchmark of its kind for human relighting, and demonstrating
state-of-the-art relighting results for novel human poses.

通过基于多视角视频的学习方法，我们提出了可重光的神经网络人体模型，能够实现真实感的光照效果、外观编辑以及任意骨骼姿势控制，并在实际场景中进行了评估，展示了针对新颖人体动作的最新重光效果。

具备内在分解和姿势控制的可重新照明的神经行为模型

Relightable Neural Actor with Intrinsic Decomposition and Pose Control

Most indoor 3D scene reconstruction methods focus on recovering 3D geometry
and scene layout. In this work, we go beyond this to propose PhotoScene, a
framework that takes input image(s) of a scene along with approximately aligned
CAD geometry (either reconstructed automatically or manually specified) and
builds a photorealistic digital twin with high-quality materials and similar
lighting. We model scene materials using procedural material graphs; such
graphs represent photorealistic and resolution-independent materials. We
optimize the parameters of these graphs and their texture scale and rotation,
as well as the scene lighting to best match the input image via a
differentiable rendering layer. We evaluate our technique on objects and layout
reconstructions from ScanNet, SUN RGB-D and stock photographs, and demonstrate
that our method reconstructs high-quality, fully relightable 3D scenes that can
be re-rendered under arbitrary viewpoints, zooms and lighting.

本文提出了一种名为 PhotoScene 的框架，可以基于输入图像及与之大致对齐的 CAD 几何形态数据（自动重建或手动指定），构建具有高质量材质和相似照明的逼真数字孪生体。该框架采用过程材料图模型建模场景材料，可以优化图参数和纹理比例、旋转，以及场景光照度以最佳匹配输入图像。文中还对该技术进行了评估，并展示了该方法对于来自 ScanNet、SUN RGB-D 和证券照片的对象和布局重建可以重渲染出高质量、可完全照明的 3D 场景。