We introduce ``Idea to Image,'' a system that enables multimodal iterative
self-refinement with GPT-4V(ision) for automatic image design and generation.
Humans can quickly identify the characteristics of different text-to-image
(T2I) models via iterative explorations. This enables them to efficiently
convert their high-level generation ideas into effective T2I prompts that can
produce good images. We investigate if systems based on large multimodal models
(LMMs) can develop analogous multimodal self-refinement abilities that enable
exploring unknown models or environments via self-refining tries. Idea2Img
cyclically generates revised T2I prompts to synthesize draft images, and
provides directional feedback for prompt revision, both conditioned on its
memory of the probed T2I model's characteristics. The iterative self-refinement
brings Idea2Img various advantages over vanilla T2I models. Notably, Idea2Img
can process input ideas with interleaved image-text sequences, follow ideas
with design instructions, and generate images of better semantic and visual
qualities. The user preference study validates the efficacy of multimodal
iterative self-refinement on automatic image design and generation.

引入了 “Idea to Image” 系统，利用 GPT-4V (Vision) 进行多模态迭代自我精炼，实现自动图像设计与生成。