The incredible generative ability of large-scale text-to-image (T2I) models
has demonstrated strong power of learning complex structures and meaningful
semantics. However, relying solely on text prompts cannot fully take advantage
of the knowledge learned by the model, especially when flexible and accurate
controlling (e.g., color and structure) is needed. In this paper, we aim to
``dig out" the capabilities that T2I models have implicitly learned, and then
explicitly use them to control the generation more granularly. Specifically, we
propose to learn simple and lightweight T2I-Adapters to align internal
knowledge in T2I models with external control signals, while freezing the
original large T2I models. In this way, we can train various adapters according
to different conditions, achieving rich control and editing effects in the
color and structure of the generation results. Further, the proposed
T2I-Adapters have attractive properties of practical value, such as
composability and generalization ability. Extensive experiments demonstrate
that our T2I-Adapter has promising generation quality and a wide range of
applications.

该论文提出了使用轻量级的 T2I 适配器来对图像生成过程进行更加精准的控制，实现在不同条件下的色彩和结构控制，并具有可组合性和广泛的应用前景。