We introduce discriminator guidance in the setting of Autoregressive
Diffusion Models. The use of a discriminator to guide a diffusion process has
previously been used for continuous diffusion models, and in this work we
derive ways of using a discriminator together with a pretrained generative
model in the discrete case. First, we show that using an optimal discriminator
will correct the pretrained model and enable exact sampling from the underlying
data distribution. Second, to account for the realistic scenario of using a
sub-optimal discriminator, we derive a sequential Monte Carlo algorithm which
iteratively takes the predictions from the discrimiator into account during the
generation process. We test these approaches on the task of generating
molecular graphs and show how the discriminator improves the generative
performance over using only the pretrained model.

在 Autoregressive Diffusion Models 中引入鉴别器指导，结合预先训练的生成模型，在离散情况下使用最优鉴别器可以纠正预训练模型并从底层数据分布中精确采样；针对使用次优鉴别器的实际情况，我们提出了一种顺序蒙特卡罗算法，在生成过程中迭代地考虑鉴别器的预测。我们在生成分子图的任务上对这些方法进行了测试，展示了鉴别器如何提高生成性能，超过仅使用预先训练的模型。

自回归扩散模型的判别指导

Discriminator Guidance for Autoregressive Diffusion Models

We introduce Autoregressive Diffusion Models (ARDMs), a model class
encompassing and generalizing order-agnostic autoregressive models (Uria et
al., 2014) and absorbing discrete diffusion (Austin et al., 2021), which we
show are special cases of ARDMs under mild assumptions. ARDMs are simple to
implement and easy to train. Unlike standard ARMs, they do not require causal
masking of model representations, and can be trained using an efficient
objective similar to modern probabilistic diffusion models that scales
favourably to highly-dimensional data. At test time, ARDMs support parallel
generation which can be adapted to fit any given generation budget. We find
that ARDMs require significantly fewer steps than discrete diffusion models to
attain the same performance. Finally, we apply ARDMs to lossless compression,
and show that they are uniquely suited to this task. Contrary to existing
approaches based on bits-back coding, ARDMs obtain compelling results not only
on complete datasets, but also on compressing single data points. Moreover,
this can be done using a modest number of network calls for (de)compression due
to the model's adaptable parallel generation.

本文介绍了 AR 模型中的一种 Autoregressive Diffusion Models，相较于其他模型具有更高的性能表现。同时，作者还将该模型应用于无损压缩，并表明该模型在此任务上也表现出了极佳的结果，其适用于数据点的压缩及适应各种生成任务。