In this paper, we present a novel two-stage approach that fully utilizes the
information provided by the reference image to establish a customized knowledge
prior for image-to-3D generation. While previous approaches primarily rely on a
general diffusion prior, which struggles to yield consistent results with the
reference image, we propose a subject-specific and multi-modal diffusion model.
This model not only aids NeRF optimization by considering the shading mode for
improved geometry but also enhances texture from the coarse results to achieve
superior refinement. Both aspects contribute to faithfully aligning the 3D
content with the subject. Extensive experiments showcase the superiority of our
method, Customize-It-3D, outperforming previous works by a substantial margin.
It produces faithful 360-degree reconstructions with impressive visual quality,
making it well-suited for various applications, including text-to-3D creation.

本文提出了一种创新的两阶段方法，充分利用参考图像提供的信息，建立用于图像到 3D 生成的定制知识先验。我们提出了一种特定于主题和多模态的扩散模型，不仅通过考虑阴影模式来改善几何形状优化 NeRF，而且从粗糙结果中增强纹理以实现更好的细化。两方面都有助于使 3D 内容与主题保持一致。广泛的实验展示了我们方法 “Customize-It-3D” 的优越性，其性能显著优于以前的工作。它产生了具有令人印象深刻的视觉质量的忠实 360 度重建，非常适用于各种应用，包括文本到 3D 创建。