Text-to-image diffusion models benefit artists with high-quality image
generation. Yet its stochastic nature prevent artists from creating consistent
images of the same character. Existing methods try to tackle this challenge and
generate consistent content in various ways. However, they either depend on
external data or require expensive tuning of the diffusion model. For this
issue, we argue that a lightweight but intricate guidance is enough to
function. Aiming at this, we lead the way to formalize the objective of
consistent generation, derive a clustering-based score function and propose a
novel paradigm, OneActor. We design a cluster-conditioned model which
incorporates posterior samples to guide the denoising trajectories towards the
target cluster. To overcome the overfitting challenge shared by one-shot tuning
pipelines, we devise auxiliary components to simultaneously augment the tuning
and regulate the inference. This technique is later verified to significantly
enhance the content diversity of generated images. Comprehensive experiments
show that our method outperforms a variety of baselines with satisfactory
character consistency, superior prompt conformity as well as high image
quality. And our method is at least 4 times faster than tuning-based baselines.
Furthermore, to our best knowledge, we first prove that the semantic space has
the same interpolation property as the latent space dose. This property can
serve as another promising tool for fine generation control.

針對文本到圖像擴散模型中的一致性生成的挑戰，提出了一種輕量級但複雜的引導方法，通過引入後驗樣本來引導去噪軌跡向目標集群集中，並設計輔助組件來同時增強調整過程和規範推理，以提高生成圖像的內容多樣性。綜合實驗結果顯示，該方法在角色一致性、提示一致性和圖像質量方面都優於基線方法，並且比基於調整的基線方法至少快 4 倍。此外，首次證明語義空間具有與潛在空間相同的插值特性，可作為另一種精細生成控制的有希望工具。