We present DiffChat, a novel method to align Large Language Models (LLMs) to
"chat" with prompt-as-input Text-to-Image Synthesis (TIS) models (e.g., Stable
Diffusion) for interactive image creation. Given a raw prompt/image and a
user-specified instruction, DiffChat can effectively make appropriate
modifications and generate the target prompt, which can be leveraged to create
the target image of high quality. To achieve this, we first collect an
instruction-following prompt engineering dataset named InstructPE for the
supervised training of DiffChat. Next, we propose a reinforcement learning
framework with the feedback of three core criteria for image creation, i.e.,
aesthetics, user preference, and content integrity. It involves an action-space
dynamic modification technique to obtain more relevant positive samples and
harder negative samples during the off-policy sampling. Content integrity is
also introduced into the value estimation function for further improvement of
produced images. Our method can exhibit superior performance than baseline
models and strong competitors based on both automatic and human evaluations,
which fully demonstrates its effectiveness.

DiffChat 是一个新颖的方法，用于将大型语言模型（LLMs）与以 prompt 为输入的文本到图像合成（TIS）模型（例如 Stable Diffusion）进行对齐，以实现交互式图像生成。它能够根据给定的原始提示 / 图像和用户指定的指令有效地进行适当的修改并生成目标提示，从而帮助生成高质量的目标图像。DiffChat 通过收集名为 InstructPE 的指令跟踪提示工程数据集进行监督训练，提出了一个包括三个核心评估标准（美学、用户喜好和内容完整性）反馈的强化学习框架来实现这一目标。在离线采样过程中，它采用一种动态修改技术来获得更相关的正样本和更难的负样本。为了进一步提高生成图像的质量，还将内容完整性引入到值估计函数中。通过比较自动评估和人工评估，我们的方法表现出优于基准模型和强竞争对手的性能，充分证明了其有效性。