This paper presents Deceptive-Human, a novel Prompt-to-NeRF framework
capitalizing state-of-the-art control diffusion models (e.g., ControlNet) to
generate a high-quality controllable 3D human NeRF. Different from direct 3D
generative approaches, e.g., DreamFusion and DreamHuman, Deceptive-Human
employs a progressive refinement technique to elevate the reconstruction
quality. This is achieved by utilizing high-quality synthetic human images
generated through the ControlNet with view-consistent loss. Our method is
versatile and readily extensible, accommodating multimodal inputs, including a
text prompt and additional data such as 3D mesh, poses, and seed images. The
resulting 3D human NeRF model empowers the synthesis of highly photorealistic
novel views from 360-degree perspectives. The key to our Deceptive-Human for
hallucinating multi-view consistent synthetic human images lies in our
progressive finetuning strategy. This strategy involves iteratively enhancing
views using the provided multimodal inputs at each intermediate step to improve
the human NeRF model. Within this iterative refinement process, view-dependent
appearances are systematically eliminated to prevent interference with the
underlying density estimation. Extensive qualitative and quantitative
experimental comparison shows that our deceptive human models achieve
state-of-the-art application quality.

利用 Deceptive-Human，这篇论文提出了一种新颖的 Prompt-to-NeRF 框架，利用最先进的控制扩散模型（例如 ControlNet）生成高质量的可控 3D 人体 NeRF，并通过逐步优化的方法提高重建质量。