We present SplatFace, a novel Gaussian splatting framework designed for 3D
human face reconstruction without reliance on accurate pre-determined geometry.
Our method is designed to simultaneously deliver both high-quality novel view
rendering and accurate 3D mesh reconstructions. We incorporate a generic 3D
Morphable Model (3DMM) to provide a surface geometric structure, making it
possible to reconstruct faces with a limited set of input images. We introduce
a joint optimization strategy that refines both the Gaussians and the morphable
surface through a synergistic non-rigid alignment process. A novel distance
metric, splat-to-surface, is proposed to improve alignment by considering both
the Gaussian position and covariance. The surface information is also utilized
to incorporate a world-space densification process, resulting in superior
reconstruction quality. Our experimental analysis demonstrates that the
proposed method is competitive with both other Gaussian splatting techniques in
novel view synthesis and other 3D reconstruction methods in producing 3D face
meshes with high geometric precision.

本文提出了一种新颖的高斯状喷洒框架 SplatFace，它用于无需准确的预定义几何形状进行 3D 人脸重建。该方法同时提供高质量的新视角渲染和精确的 3D 网格重建，通过引入通用的 3D 可塑模型（3DMM）来提供表面几何结构，从而使得以有限的图片集合进行人脸重建成为可能。作者提出了一种联合优化策略，通过协同非刚性对准过程来改善高斯函数和可塑性表面。还提出了一种新的距离度量方法，即喷洒 - 表面距离，通过考虑高斯位置和协方差来改善对准。利用表面信息还用于引入世界空间的稠密化过程，从而获得更好的重建质量。实验分析表明，该方法在新视角合成的高斯状喷洒技术和 3D 重建方法中均具有竞争力，能够生成具有高几何精度的 3D 人脸网格。

SplatFace: 高斯分布面面片重建及其可优化表面

SplatFace: Gaussian Splat Face Reconstruction Leveraging an Optimizable  Surface

In this work we propose a novel model-based deep convolutional autoencoder
that addresses the highly challenging problem of reconstructing a 3D human face
from a single in-the-wild color image. To this end, we combine a convolutional
encoder network with an expert-designed generative model that serves as
decoder. The core innovation is our new differentiable parametric decoder that
encapsulates image formation analytically based on a generative model. Our
decoder takes as input a code vector with exactly defined semantic meaning that
encodes detailed face pose, shape, expression, skin reflectance and scene
illumination. Due to this new way of combining CNN-based with model-based face
reconstruction, the CNN-based encoder learns to extract semantically meaningful
parameters from a single monocular input image. For the first time, a CNN
encoder and an expert-designed generative model can be trained end-to-end in an
unsupervised manner, which renders training on very large (unlabeled) real
world data feasible. The obtained reconstructions compare favorably to current
state-of-the-art approaches in terms of quality and richness of representation.

本文提出了一种基于深度卷积自编码器的新颖模型，用于从野外拍摄的单幅彩色图像重建三维人脸。通过将卷积编码器与专家设计的生成模型相结合，提出了一种新的可微分参数解码器。该解码器将基于生成式模型的图像形成解析包括在内，以输入的码向量作为输入，从单个单眼输入图像中提取具有明确定义语义含义的参数。该文章的一个突破是在无监督的情况下，首次实现了 CNN 编码器和专家设计的生成模型的端到端训练，这使得对非常大的（未标记的）真实世界数据进行训练成为可能。所得到的重建结果在质量和表示的丰富性方面均优于当前最先进的方法。