Recent advancements in large scale text-to-image models have opened new
possibilities for guiding the creation of images through human-devised natural
language. However, while prior literature has primarily focused on the
generation of individual images, it is essential to consider the capability of
these models to ensure coherency within a sequence of images to fulfill the
demands of real-world applications such as storytelling. To address this, here
we present a novel neural pipeline for generating a coherent storybook from the
plain text of a story. Specifically, we leverage a combination of a pre-trained
Large Language Model and a text-guided Latent Diffusion Model to generate
coherent images. While previous story synthesis frameworks typically require a
large-scale text-to-image model trained on expensive image-caption pairs to
maintain the coherency, we employ simple textual inversion techniques along
with detector-based semantic image editing which allows zero-shot generation of
the coherent storybook. Experimental results show that our proposed method
outperforms state-of-the-art image editing baselines.

本文提出了一种基于文本的神经管道，利用预训练的语言模型和文本引导的潜在扩散模型来生成一系列连贯的图像，实现连贯的故事书生成，并展示了无需昂贵的图像 - 标题对训练的文本转换技术的性能表现。