Token-based masked generative models are gaining popularity for their fast
inference time with parallel decoding. While recent token-based approaches
achieve competitive performance to diffusion-based models, their generation
performance is still suboptimal as they sample multiple tokens simultaneously
without considering the dependence among them. We empirically investigate this
problem and propose a learnable sampling model, Text-Conditioned Token
Selection (TCTS), to select optimal tokens via localized supervision with text
information. TCTS improves not only the image quality but also the semantic
alignment of the generated images with the given texts. To further improve the
image quality, we introduce a cohesive sampling strategy, Frequency Adaptive
Sampling (FAS), to each group of tokens divided according to the self-attention
maps. We validate the efficacy of TCTS combined with FAS with various
generative tasks, demonstrating that it significantly outperforms the baselines
in image-text alignment and image quality. Our text-conditioned sampling
framework further reduces the original inference time by more than 50% without
modifying the original generative model.

本文提出了一种学习可采样模型的 Text-Conditioned Token Selection（TCTS）方案，通过文本信息的本地化监督来选择最优选项，这样可以提高生成图像的质量和与给定文本的语义对齐度，并引入一种统一的采样策略 Frequency Adaptive Sampling（FAS），以进一步提高图像质量和文本对齐度。我们在各种生成任务中验证了 TCTS 与 FAS 的功效，并展示它在图像 - 文本对齐和图像质量方面显著优于基线，文本条件采样框架可以将原始推理时间缩短 50% 以上，这是继承基线模型的一个重大优势。