We propose MVDream, a multi-view diffusion model that is able to generate
geometrically consistent multi-view images from a given text prompt. By
leveraging image diffusion models pre-trained on large-scale web datasets and a
multi-view dataset rendered from 3D assets, the resulting multi-view diffusion
model can achieve both the generalizability of 2D diffusion and the consistency
of 3D data. Such a model can thus be applied as a multi-view prior for 3D
generation via Score Distillation Sampling, where it greatly improves the
stability of existing 2D-lifting methods by solving the 3D consistency problem.
Finally, we show that the multi-view diffusion model can also be fine-tuned
under a few shot setting for personalized 3D generation, i.e. DreamBooth3D
application, where the consistency can be maintained after learning the subject
identity.

我们提出 MVDream，这是一个多视图扩散模型，能够根据给定的文本提示生成几何一致的多视图图像。通过利用在大规模 Web 数据集上预训练的图像扩散模型和从 3D 资源渲染的多视图数据集，所得到的多视图扩散模型可以实现 2D 扩散的概括性和 3D 数据的一致性。该模型可以被应用为 3D 生成的多视图先验，通过 Score Distillation Sampling 来解决现有 2D-lifting 方法中的 3D 一致性问题，从而极大地提高了稳定性。最后，我们展示了多视图扩散模型也可以在少量样本设置下进行微调，用于个性化的 3D 生成，即 DreamBooth3D 应用中，在学习主体身份之后可以保持一致性。