Current controls over diffusion models (e.g., through text or ControlNet) for
image generation fall short in recognizing abstract, continuous attributes like
illumination direction or non-rigid shape change. In this paper, we present an
approach for allowing users of text-to-image models to have fine-grained
control of several attributes in an image. We do this by engineering special
sets of input tokens that can be transformed in a continuous manner -- we call
them Continuous 3D Words. These attributes can, for example, be represented as
sliders and applied jointly with text prompts for fine-grained control over
image generation. Given only a single mesh and a rendering engine, we show that
our approach can be adopted to provide continuous user control over several
3D-aware attributes, including time-of-day illumination, bird wing orientation,
dollyzoom effect, and object poses. Our method is capable of conditioning image
creation with multiple Continuous 3D Words and text descriptions simultaneously
while adding no overhead to the generative process. Project Page:
this https URL

本文介绍了一种使文本生成图像的模型能够识别抽象连续属性的方法，通过构建特殊的输入标记集，可以连续地转换图像属性，从而实现用户对图像生成过程的细粒度控制，包括时间和方向的光照、鸟翼方向、摄像机特效以及物体姿态等等。