Conventional wisdom holds that autoregressive models for image generation are
typically accompanied by vector-quantized tokens. We observe that while a
discrete-valued space can facilitate representing a categorical distribution,
it is not a necessity for autoregressive modeling. In this work, we propose to
model the per-token probability distribution using a diffusion procedure, which
allows us to apply autoregressive models in a continuous-valued space. Rather
than using categorical cross-entropy loss, we define a Diffusion Loss function
to model the per-token probability. This approach eliminates the need for
discrete-valued tokenizers. We evaluate its effectiveness across a wide range
of cases, including standard autoregressive models and generalized masked
autoregressive (MAR) variants. By removing vector quantization, our image
generator achieves strong results while enjoying the speed advantage of
sequence modeling. We hope this work will motivate the use of autoregressive
generation in other continuous-valued domains and applications.

我们提出了使用扩散过程模型每个令牌的概率分布，从而可以将自回归模型应用于连续值空间的方法，并通过定义扩散损失函数来替代离散化的令牌化。通过消除向量量化，我们的图像生成器在享受序列建模的速度优势的同时取得了强大的结果，并希望该方法能促进在其他连续值领域和应用中使用自回归生成。