Nearly every recent image synthesis approach, including diffusion,
masked-token prediction, and next-token prediction, uses a Transformer network
architecture. Despite this common backbone, there has been no direct, compute
controlled comparison of how these approaches affect performance and
efficiency. We analyze the scalability of each approach through the lens of
compute budget measured in FLOPs. We find that token prediction methods, led by
next-token prediction, significantly outperform diffusion on prompt following.
On image quality, while next-token prediction initially performs better,
scaling trends suggest it is eventually matched by diffusion. We compare the
inference compute efficiency of each approach and find that next token
prediction is by far the most efficient. Based on our findings we recommend
diffusion for applications targeting image quality and low latency; and
next-token prediction when prompt following or throughput is more important.

最近的图像合成方法主要使用 Transformer 网络架构，本研究通过计算预算的可扩展性对这些方法进行比较，发现基于标记预测的方法在提示后续上明显优于扩散，但在图像质量上，扩散表现与扩大趋势相匹配。论文还比较了每种方法的推理计算效率，发现基于标记预测的方法是最高效的。根据研究发现，建议在追随提示或吞吐量更重要时使用标记预测，而在追求图像质量和低延迟的应用场景中使用扩散。