Despite the growing success of diffusion models in continuous-valued domains
(e.g., images), diffusion-based language models on discrete text have yet to
match autoregressive language models on text generation benchmarks. In this
work, we present SSD-LM -- a diffusion language model with two key design
choices. First, SSD-LM is semi-autoregressive, iteratively generating blocks of
text, allowing for flexible output length at decoding time while enabling local
bidirectional context updates. Second, it is simplex-based, performing
diffusion on the natural vocabulary space rather than a learned latent space,
allowing us to incorporate classifier guidance and modular control without any
adaptation of off-the-shelf classifiers. We evaluate SSD-LM on unconstrained as
well as controlled text generation benchmarks, and show that it matches or
outperforms strong autoregressive GPT-2 baselines across standard quality and
diversity metrics.

本论文介绍了 SSD-LM - 一种扩散语言模型，它是半自回归的，可以迭代地生成文本块，在不适用 GPT-2 后，它在标准的质量和多样性指标上与 strong autoregressive 基线匹配或优于其性能