Discrete flow-based models are a recently proposed class of generative models
that learn invertible transformations for discrete random variables. Since they
do not require data dequantization and maximize an exact likelihood objective,
they can be used in a straight-forward manner for lossless compression. In this
paper, we introduce a new discrete flow-based model for categorical random
variables: Discrete Denoising Flows (DDFs). In contrast with other discrete
flow-based models, our model can be locally trained without introducing
gradient bias. We show that DDFs outperform Discrete Flows on modeling a toy
example, binary MNIST and Cityscapes segmentation maps, measured in
log-likelihood.

本研究提出了离散去噪流模型（Discrete Denoising Flows, DDFs），用于建模离散的随机变量，并证明在对数似然度量中，在建模玩具示例，二进制 MNIST 和 Cityscapes 分割地图方面，DDFs 优于 Discrete Flows。

离散去噪流

Discrete Denoising Flows

Neural text generation is a key tool in natural language applications, but it
is well known there are major problems at its core. In particular, standard
likelihood training and decoding leads to dull and repetitive outputs. While
some post-hoc fixes have been proposed, in particular top-$k$ and nucleus
sampling, they do not address the fact that the token-level probabilities
predicted by the model are poor. In this paper we show that the likelihood
objective itself is at fault, resulting in a model that assigns too much
probability to sequences containing repeats and frequent words, unlike those
from the human training distribution. We propose a new objective, unlikelihood
training, which forces unlikely generations to be assigned lower probability by
the model. We show that both token and sequence level unlikelihood training
give less repetitive, less dull text while maintaining perplexity, giving
superior generations using standard greedy or beam search. According to human
evaluations, our approach with standard beam search also outperforms the
currently popular decoding methods of nucleus sampling or beam blocking, thus
providing a strong alternative to existing techniques.

提出一种基于非似然训练的神经文本生成方法，有效降低生成文本的重复性，将标准束搜索的输出性能提升至目前最优，并提供了一种强有力的替代神经文本生成中已有技术的方法。