Although autoregressive models have achieved promising results on image
generation, their unidirectional generation process prevents the resultant
images from fully reflecting global contexts. To address the issue, we propose
an effective image generation framework of Draft-and-Revise with Contextual
RQ-transformer to consider global contexts during the generation process. As a
generalized VQ-VAE, RQ-VAE first represents a high-resolution image as a
sequence of discrete code stacks. After code stacks in the sequence are
randomly masked, Contextual RQ-Transformer is trained to infill the masked code
stacks based on the unmasked contexts of the image. Then, Contextual
RQ-Transformer uses our two-phase decoding, Draft-and-Revise, and generates an
image, while exploiting the global contexts of the image during the generation
process. Specifically. in the draft phase, our model first focuses on
generating diverse images despite rather low quality. Then, in the revise
phase, the model iteratively improves the quality of images, while preserving
the global contexts of generated images. In experiments, our method achieves
state-of-the-art results on conditional image generation. We also validate that
the Draft-and-Revise decoding can achieve high performance by effectively
controlling the quality-diversity trade-off in image generation.

该研究提出了一种有效的基于文本上下文和全局语境的图像生成框架，名为 Draft-and-Revise with Contextual RQ-transformer，通过 Contextual RQ-transformer 考虑图像全局上下文信息以最大程度地反映全局语境。在实验中，我们的方法在条件图像生成上达到了最先进的效果。

草稿与修订：基于上下文 RQ-Transformer 的有效图像生成

Draft-and-Revise: Effective Image Generation with Contextual  RQ-Transformer

For autoregressive (AR) modeling of high-resolution images, vector
quantization (VQ) represents an image as a sequence of discrete codes. A short
sequence length is important for an AR model to reduce its computational costs
to consider long-range interactions of codes. However, we postulate that
previous VQ cannot shorten the code sequence and generate high-fidelity images
together in terms of the rate-distortion trade-off. In this study, we propose
the two-stage framework, which consists of Residual-Quantized VAE (RQ-VAE) and
RQ-Transformer, to effectively generate high-resolution images. Given a fixed
codebook size, RQ-VAE can precisely approximate a feature map of an image and
represent the image as a stacked map of discrete codes. Then, RQ-Transformer
learns to predict the quantized feature vector at the next position by
predicting the next stack of codes. Thanks to the precise approximation of
RQ-VAE, we can represent a 256$\times$256 image as 8$\times$8 resolution of the
feature map, and RQ-Transformer can efficiently reduce the computational costs.
Consequently, our framework outperforms the existing AR models on various
benchmarks of unconditional and conditional image generation. Our approach also
has a significantly faster sampling speed than previous AR models to generate
high-quality images.

本文提出了一种两阶段的框架，由残差量化 VAE（RQ-VAE）和 RQ-Transformer 组成，旨在有效地生成高分辨率图像，通过精确的逼近，我们可以将 256×256 像素的图像表示为 8×8 特征映射，并且 RQ-Transformer 可以在保证生成高品质图像的同时减少计算成本。