Existing Score Distillation Sampling (SDS)-based methods have driven significant progress in text-to-3D generation. However, 3D models produced by SDS-based methods tend to exhibit over-smoothing and low-quality outputs. These issues arise from the mode-seeking behavior of current methods, where the scores used to update the model oscillate between multiple modes, resulting in unstable optimization and diminished output quality. To address this problem, we introduce a novel image prompt score distillation loss named ISD, which employs a reference image to direct text-to-3D optimization toward a specific mode. Our ISD loss can be implemented by using IP-Adapter, a lightweight adapter for integrating image prompt capability to a text-to-image diffusion model, as a mode-selection module. A variant of this adapter, when not being prompted by a reference image, can serve as an efficient control variate to reduce variance in score estimates, thereby enhancing both output quality and optimization stability. Our experiments demonstrate that the ISD loss consistently achieves visually coherent, high-quality outputs and improves optimization speed compared to prior text-to-3D methods, as demonstrated through both qualitative and quantitative evaluations on the T3Bench benchmark suite.

本研究针对现有基于评分蒸馏采样(SDS)方法在文本到3D生成中的输出过于平滑和低质量的问题，提出了一种新颖的图像提示评分蒸馏损失ISD。该方法通过参考图像引导模型优化，显著提升了输出质量和优化稳定性，实验结果表明相较于以往方法，ISD损失在生成的视觉一致性和质量上有显著改善。

ModeDreamer：基于参考图像提示的文本到3D生成的模式引导评分蒸馏