At the core of portrait photography is the search for ideal lighting and
viewpoint. The process often requires advanced knowledge in photography and an
elaborate studio setup. In this work, we propose Holo-Relighting, a volumetric
relighting method that is capable of synthesizing novel viewpoints, and novel
lighting from a single image. Holo-Relighting leverages the pretrained 3D GAN
(EG3D) to reconstruct geometry and appearance from an input portrait as a set
of 3D-aware features. We design a relighting module conditioned on a given
lighting to process these features, and predict a relit 3D representation in
the form of a tri-plane, which can render to an arbitrary viewpoint through
volume rendering. Besides viewpoint and lighting control, Holo-Relighting also
takes the head pose as a condition to enable head-pose-dependent lighting
effects. With these novel designs, Holo-Relighting can generate complex
non-Lambertian lighting effects (e.g., specular highlights and cast shadows)
without using any explicit physical lighting priors. We train Holo-Relighting
with data captured with a light stage, and propose two data-rendering
techniques to improve the data quality for training the volumetric relighting
system. Through quantitative and qualitative experiments, we demonstrate
Holo-Relighting can achieve state-of-the-arts relighting quality with better
photorealism, 3D consistency and controllability.

Holo-Relighting 是一种能够从单张图像中合成新视角和新光照的体积照明方法，利用预训练的 3D GAN 来重建几何和外观，并通过体积渲染将其渲染到任意观点。通过图像渲染和质量提升技术，Holo-Relighting 可以实现更好的光照效果、3D 一致性和可控性，从而达到最先进的照明质量。

全息照明：从单张图像实现可控体积肖像照明

Holo-Relighting: Controllable Volumetric Portrait Relighting from a  Single Image

3D facial avatar reconstruction has been a significant research topic in
computer graphics and computer vision, where photo-realistic rendering and
flexible controls over poses and expressions are necessary for many related
applications. Recently, its performance has been greatly improved with the
development of neural radiance fields (NeRF). However, most existing NeRF-based
facial avatars focus on subject-specific reconstruction and reenactment,
requiring multi-shot images containing different views of the specific subject
for training, and the learned model cannot generalize to new identities,
limiting its further applications. In this work, we propose a one-shot 3D
facial avatar reconstruction framework that only requires a single source image
to reconstruct a high-fidelity 3D facial avatar. For the challenges of lacking
generalization ability and missing multi-view information, we leverage the
generative prior of 3D GAN and develop an efficient encoder-decoder network to
reconstruct the canonical neural volume of the source image, and further
propose a compensation network to complement facial details. To enable
fine-grained control over facial dynamics, we propose a deformation field to
warp the canonical volume into driven expressions. Through extensive
experimental comparisons, we achieve superior synthesis results compared to
several state-of-the-art methods.

提出了一种只需要一张源图像就能重建高保真度的三维面部角色建模方法，通过使用 3D GAN 的生成先验和有效的编码器 - 解码器网络来重建源图像的规范神经体积，同时使用补偿网络来补充面部细节，并引入变形场来对面部表情进行精细控制。与多种先进方法相比，实验结果显示合成效果更加优越。

NOFA：基于 NeRF 的一次性面部头像重建

NOFA: NeRF-based One-shot Facial Avatar Reconstruction

Recovering a textured 3D mesh from a monocular image is highly challenging,
particularly for in-the-wild objects that lack 3D ground truths. In this work,
we present MeshInversion, a novel framework to improve the reconstruction by
exploiting the generative prior of a 3D GAN pre-trained for 3D textured mesh
synthesis. Reconstruction is achieved by searching for a latent space in the 3D
GAN that best resembles the target mesh in accordance with the single view
observation. Since the pre-trained GAN encapsulates rich 3D semantics in terms
of mesh geometry and texture, searching within the GAN manifold thus naturally
regularizes the realness and fidelity of the reconstruction. Importantly, such
regularization is directly applied in the 3D space, providing crucial guidance
of mesh parts that are unobserved in the 2D space. Experiments on standard
benchmarks show that our framework obtains faithful 3D reconstructions with
consistent geometry and texture across both observed and unobserved parts.
Moreover, it generalizes well to meshes that are less commonly seen, such as
the extended articulation of deformable objects. Code is released at
this https URL

通过利用预先训练的 3D GAN 生成先验来改善重建结果，本文提出了一种名为 MeshInversion 的新型框架，它在单视角观察下通过搜索最相似目标网格的 3D GAN 的潜在空间来实现重建，从而在 3D 空间中直接应用正则化，提供了未观测的网格部位的关键指导，实现了出色的 3D 重建。