Accurate 3D face shape estimation is an enabling technology with applications
in healthcare, security, and creative industries, yet current state-of-the-art
methods either rely on self-supervised training with 2D image data or
supervised training with very limited 3D data. To bridge this gap, we present a
novel approach which uses a conditioned stable diffusion model for face image
generation, leveraging the abundance of 2D facial information to inform 3D
space. By conditioning stable diffusion on depth maps sampled from a 3D
Morphable Model (3DMM) of the human face, we generate diverse and
shape-consistent images, forming the basis of SynthFace. We introduce this
large-scale synthesised dataset of 250K photorealistic images and corresponding
3DMM parameters. We further propose ControlFace, a deep neural network, trained
on SynthFace, which achieves competitive performance on the NoW benchmark,
without requiring 3D supervision or manual 3D asset creation.

通过在 3DMM 的深度图中采样生成的稳定扩散模型对人脸图像进行生成，利用丰富的 2D 面部信息提供 3D 空间信息，形成 SynthFace，并在此基础上训练的神经网络 ControlFace，在 NoW 基准测试上实现了竞争性能。