Our paper presents My3DGen, a practical system for creating a personalized
and lightweight 3D generative prior using as few as 10 images. My3DGen can
reconstruct multi-view consistent images from an input test image, and generate
novel appearances by interpolating between any two images of the same
individual. While recent studies have demonstrated the effectiveness of
personalized generative priors in producing high-quality 2D portrait
reconstructions and syntheses, to the best of our knowledge, we are the first
to develop a personalized 3D generative prior. Instead of fine-tuning a large
pre-trained generative model with millions of parameters to achieve
personalization, we propose a parameter-efficient approach. Our method involves
utilizing a pre-trained model with fixed weights as a generic prior, while
training a separate personalized prior through low-rank decomposition of the
weights in each convolution and fully connected layer. However,
parameter-efficient few-shot fine-tuning on its own often leads to overfitting.
To address this, we introduce a regularization technique based on symmetry of
human faces. This regularization enforces that novel view renderings of a
training sample, rendered from symmetric poses, exhibit the same identity. By
incorporating this symmetry prior, we enhance the quality of reconstruction and
synthesis, particularly for non-frontal (profile) faces. Our final system
combines low-rank fine-tuning with symmetry regularization and significantly
surpasses the performance of pre-trained models, e.g. EG3D. It introduces only
approximately 0.6 million additional parameters per identity compared to 31
million for full finetuning of the original model. As a result, our system
achieves a 50-fold reduction in model size without sacrificing the quality of
the generated 3D faces. Code will be available at our project page:
this https URL

My3DGen 是一个实用系统，使用不超过 10 张图像创建个性化和轻量级的 3D 生成先验系统。该系统利用预训练模型的固定权重作为通用先验，通过低秩分解在每个卷积和完全连接层的权重上进行单独的个性化先验训练，并通过基于人脸对称的规则化技术调整网络，指导其学习参数并减少过拟合。该系统仅增加了与完全微调的原始模型相比，每个身份约 0.6 百万个附加参数，同时保持生成的 3D 人脸的质量。