Consistency Models (CMs) have showed a promise in creating visual content
efficiently and with high quality. However, the way to add new conditional
controls to the pretrained CMs has not been explored. In this technical report,
we consider alternative strategies for adding ControlNet-like conditional
control to CMs and present three significant findings. 1) ControlNet trained
for diffusion models (DMs) can be directly applied to CMs for high-level
semantic controls but struggles with low-level detail and realism control. 2)
CMs serve as an independent class of generative models, based on which
ControlNet can be trained from scratch using Consistency Training proposed by
Song et al. 3) A lightweight adapter can be jointly optimized under multiple
conditions through Consistency Training, allowing for the swift transfer of
DMs-based ControlNet to CMs. We study these three solutions across various
conditional controls, including edge, depth, human pose, low-resolution image
and masked image with text-to-image latent consistency models.

通过采用三种不同的策略，在现有的一致性模型中添加类似 ControlNet 的条件控制，即使在低层细节和真实感控制方面依然困难，将 ControlNet 直接应用于高级语义控制的一致性模型，并通过 Song 等人提出的一致性训练从头开始训练基于 Consistency Models 的 ControlNet，通过一致性训练可以同时优化轻量级适配器，从而实现 DMs-based ControlNet 的快速迁移到 CMs。

CCM：为文本到图像一致性模型添加条件控制

CCM: Adding Conditional Controls to Text-to-Image Consistency Models

This paper explores promptable NeRF generation (e.g., text prompt or single
image prompt) for direct conditioning and fast generation of NeRF parameters
for the underlying 3D scenes, thus undoing complex intermediate steps while
providing full 3D generation with conditional control. Unlike previous
diffusion-CLIP-based pipelines that involve tedious per-prompt optimizations,
Prompt2NeRF-PIL is capable of generating a variety of 3D objects with a single
forward pass, leveraging a pre-trained implicit latent space of NeRF
parameters. Furthermore, in zero-shot tasks, our experiments demonstrate that
the NeRFs produced by our method serve as semantically informative
initializations, significantly accelerating the inference process of existing
prompt-to-NeRF methods. Specifically, we will show that our approach speeds up
the text-to-NeRF model DreamFusion and the 3D reconstruction speed of the
image-to-NeRF method Zero-1-to-3 by 3 to 5 times.

本研究探讨了通过文本提示或单个图像提示进行的可提示的 NeRF 生成，以直接调节和快速生成底层 3D 场景的 NeRF 参数，从而取消复杂的中间步骤并提供具有条件控制的完整的 3D 生成。