Reconstructing an avatar from a portrait image has many applications in
multimedia, but remains a challenging research problem. Extracting reflectance
maps and geometry from one image is ill-posed: recovering geometry is a
one-to-many mapping problem and reflectance and light are difficult to
disentangle. Accurate geometry and reflectance can be captured under the
controlled conditions of a light stage, but it is costly to acquire large
datasets in this fashion. Moreover, training solely with this type of data
leads to poor generalization with in-the-wild images. This motivates the
introduction of MoSAR, a method for 3D avatar generation from monocular images.
We propose a semi-supervised training scheme that improves generalization by
learning from both light stage and in-the-wild datasets. This is achieved using
a novel differentiable shading formulation. We show that our approach
effectively disentangles the intrinsic face parameters, producing relightable
avatars. As a result, MoSAR estimates a richer set of skin reflectance maps,
and generates more realistic avatars than existing state-of-the-art methods. We
also introduce a new dataset, named FFHQ-UV-Intrinsics, the first public
dataset providing intrisic face attributes at scale (diffuse, specular, ambient
occlusion and translucency maps) for a total of 10k subjects. The project
website and the dataset are available on the following link:
this https URL

基于 MoSAR 方法，从单目图像中生成 3D 头像。我们提出了一种半监督训练方案，通过同时学习光型平台和野外数据集来提高泛化性能，并利用一种新颖的可微分着色公式有效地分离面部参数，生成逼真且可重光的人物头像。此外，我们还推出了一个新数据集 FFHQ-UV-Intrinsics，该数据集提供了 10k 个主题的面部属性，包括漫反射、镜面反射、环境光遮蔽和透明度贴图，是首个公开的大规模数据集。

MoSAR: 基于单目半监督模型的头像重建方法，利用可微分着色

MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using  Differentiable Shading

How much does a single image reveal about the environment it was taken in? In
this paper, we investigate how much of that information can be retrieved from a
foreground object, combined with the background (i.e. the visible part of the
environment). Assuming it is not perfectly diffuse, the foreground object acts
as a complexly shaped and far-from-perfect mirror. An additional challenge is
that its appearance confounds the light coming from the environment with the
unknown materials it is made of. We propose a learning-based approach to
predict the environment from multiple reflectance maps that are computed from
approximate surface normals. The proposed method allows us to jointly model the
statistics of environments and material properties. We train our system from
synthesized training data, but demonstrate its applicability to real-world
data. Interestingly, our analysis shows that the information obtained from
objects made out of multiple materials often is complementary and leads to
better performance.

该研究论文利用基于学习的方法和多个反射率图预测环境和物质特性，从而揭示前景物体和背景之间的关系及信息获取方式。