At the core of portrait photography is the search for ideal lighting and
viewpoint. The process often requires advanced knowledge in photography and an
elaborate studio setup. In this work, we propose Holo-Relighting, a volumetric
relighting method that is capable of synthesizing novel viewpoints, and novel
lighting from a single image. Holo-Relighting leverages the pretrained 3D GAN
(EG3D) to reconstruct geometry and appearance from an input portrait as a set
of 3D-aware features. We design a relighting module conditioned on a given
lighting to process these features, and predict a relit 3D representation in
the form of a tri-plane, which can render to an arbitrary viewpoint through
volume rendering. Besides viewpoint and lighting control, Holo-Relighting also
takes the head pose as a condition to enable head-pose-dependent lighting
effects. With these novel designs, Holo-Relighting can generate complex
non-Lambertian lighting effects (e.g., specular highlights and cast shadows)
without using any explicit physical lighting priors. We train Holo-Relighting
with data captured with a light stage, and propose two data-rendering
techniques to improve the data quality for training the volumetric relighting
system. Through quantitative and qualitative experiments, we demonstrate
Holo-Relighting can achieve state-of-the-arts relighting quality with better
photorealism, 3D consistency and controllability.

Holo-Relighting 是一种能够从单张图像中合成新视角和新光照的体积照明方法，利用预训练的 3D GAN 来重建几何和外观，并通过体积渲染将其渲染到任意观点。通过图像渲染和质量提升技术，Holo-Relighting 可以实现更好的光照效果、3D 一致性和可控性，从而达到最先进的照明质量。

全息照明：从单张图像实现可控体积肖像照明

Holo-Relighting: Controllable Volumetric Portrait Relighting from a  Single Image

Casually-taken portrait photographs often suffer from unflattering lighting
and shadowing because of suboptimal conditions in the environment. Aesthetic
qualities such as the position and softness of shadows and the lighting ratio
between the bright and dark parts of the face are frequently determined by the
constraints of the environment rather than by the photographer. Professionals
address this issue by adding light shaping tools such as scrims, bounce cards,
and flashes. In this paper, we present a computational approach that gives
casual photographers some of this control, thereby allowing poorly-lit
portraits to be relit post-capture in a realistic and easily-controllable way.
Our approach relies on a pair of neural networks---one to remove foreign
shadows cast by external objects, and another to soften facial shadows cast by
the features of the subject and to add a synthetic fill light to improve the
lighting ratio. To train our first network we construct a dataset of real-world
portraits wherein synthetic foreign shadows are rendered onto the face, and we
show that our network learns to remove those unwanted shadows. To train our
second network we use a dataset of Light Stage scans of human subjects to
construct input/output pairs of input images harshly lit by a small light
source, and variably softened and fill-lit output images of each face. We
propose a way to explicitly encode facial symmetry and show that our dataset
and training procedure enable the model to generalize to images taken in the
wild. Together, these networks enable the realistic and aesthetically pleasing
enhancement of shadows and lights in real-world portrait images

本文提出了一个计算方法，通过使用神经网络去除外部阴影并软化面部阴影，同时添加合成填充光线，从而使拍摄不佳的人像照片得以以逼真和易于控制的方式进行重新照明。