Text-guided image generation has progressed rapidly in recent years,
inspiring major breakthroughs in text-guided shape generation. Recently, it has
been shown that using score distillation, one can successfully text-guide a
NeRF model to generate a 3D object. We adapt the score distillation to the
publicly available, and computationally efficient, Latent Diffusion Models,
which apply the entire diffusion process in a compact latent space of a
pretrained autoencoder. As NeRFs operate in image space, a naive solution for
guiding them with latent score distillation would require encoding to the
latent space at each guidance step. Instead, we propose to bring the NeRF to
the latent space, resulting in a Latent-NeRF. Analyzing our Latent-NeRF, we
show that while Text-to-3D models can generate impressive results, they are
inherently unconstrained and may lack the ability to guide or enforce a
specific 3D structure. To assist and direct the 3D generation, we propose to
guide our Latent-NeRF using a Sketch-Shape: an abstract geometry that defines
the coarse structure of the desired object. Then, we present means to integrate
such a constraint directly into a Latent-NeRF. This unique combination of text
and shape guidance allows for increased control over the generation process. We
also show that latent score distillation can be successfully applied directly
on 3D meshes. This allows for generating high-quality textures on a given
geometry. Our experiments validate the power of our different forms of guidance
and the efficiency of using latent rendering. Implementation is available at
this https URL

本文介绍了如何采用分数蒸馏技术，在以图像为操作空间的 NeRF 模型中利用潜空间分数生成三维对象。同时，还提出了一种利用草图风格引导的 Latent-NeRF 模型，增加了对生成过程的控制能力，并在此基础上进一步将文本和草图风格引导结合，为生成高质量纹理的三维对象提供了有效的实现方法。