Modern text-to-image generation models produce high-quality images that are
both photorealistic and faithful to the text prompts. However, this quality
comes at significant computational cost: nearly all of these models are
iterative and require running inference multiple times with large models. This
iterative process is needed to ensure that different regions of the image are
not only aligned with the text prompt, but also compatible with each other. In
this work, we propose a light-weight approach to achieving this compatibility
between different regions of an image, using a Markov Random Field (MRF) model.
This method is shown to work in conjunction with the recently proposed Muse
model. The MRF encodes the compatibility among image tokens at different
spatial locations and enables us to significantly reduce the required number of
Muse prediction steps. Inference with the MRF is significantly cheaper, and its
parameters can be quickly learned through back-propagation by modeling MRF
inference as a differentiable neural-network layer. Our full model, SPEGTI,
uses this proposed MRF model to speed up Muse by 1.5X with no loss in output
image quality.

通过使用马尔可夫随机场模型，我们提出了一个轻量级方法来实现图像不同区域之间的兼容性，从而在不降低输出图像质量的情况下提高了图像到文本生成模型的速度。