The field of image synthesis is currently flourishing due to the advancements
in diffusion models. While diffusion models have been successful, their
computational intensity has prompted the pursuit of more efficient
alternatives. As a representative work, non-autoregressive Transformers (NATs)
have been recognized for their rapid generation. However, a major drawback of
these models is their inferior performance compared to diffusion models. In
this paper, we aim to re-evaluate the full potential of NATs by revisiting the
design of their training and inference strategies. Specifically, we identify
the complexities in properly configuring these strategies and indicate the
possible sub-optimality in existing heuristic-driven designs. Recognizing this,
we propose to go beyond existing methods by directly solving the optimal
strategies in an automatic framework. The resulting method, named AutoNAT,
advances the performance boundaries of NATs notably, and is able to perform
comparably with the latest diffusion models at a significantly reduced
inference cost. The effectiveness of AutoNAT is validated on four benchmark
datasets, i.e., ImageNet-256 & 512, MS-COCO, and CC3M. Our code is available at
this https URL

通过重新评估非自回归变压器的训练和推理策略设计，本文提出了一种名为 AutoNAT 的方法，该方法在自动框架中直接解决最优策略，从而大幅提升了非自回归变压器的性能，并且能够与最新的扩散模型在显著降低推理成本的同时进行可比较的表现。