Recent advancements in text-to-image generation have revolutionized numerous
fields, including art and cinema, by automating the generation of high-quality,
context-aware images and video. However, the utility of these technologies is
often limited by the inadequacy of text prompts in guiding the generator to
produce artistically coherent and subject-relevant images. In this paper, We
describe the techniques that can be used to make Large Language Models (LLMs)
act as Art Directors that enhance image and video generation. We describe our
unified system for this called "LaDi". We explore how LaDi integrates multiple
techniques for augmenting the capabilities of text-to-image generators (T2Is)
and text-to-video generators (T2Vs), with a focus on constrained decoding,
intelligent prompting, fine-tuning, and retrieval. LaDi and these techniques
are being used today in apps and platforms developed by Plai Labs.

近期的文本到图像生成的进展在艺术和电影等许多领域引起了革命，通过自动化生成高质量、上下文感知的图像和视频。本文介绍了一种技术，可以使大型语言模型作为艺术总监来增强图像和视频生成。我们描述了我们称之为 “LaDi” 的这种统一系统。我们探讨了 LaDi 如何集成多种技术来增加文本到图像生成器和文本到视频生成器的能力，重点是受限解码、智能提示、微调和检索。LaDi 和这些技术目前正在由 Plai Labs 开发的应用和平台中使用。