We introduce a novel generative model, the Discrete Distribution Networks
(DDN), that approximates data distribution using hierarchical discrete
distributions. We posit that since the features within a network inherently
contain distributional information, liberating the network from a single output
to concurrently generate multiple samples proves to be highly effective.
Therefore, DDN fits the target distribution, including continuous ones, by
generating multiple discrete sample points. To capture finer details of the
target data, DDN selects the output that is closest to the Ground Truth (GT)
from the coarse results generated in the first layer. This selected output is
then fed back into the network as a condition for the second layer, thereby
generating new outputs more similar to the GT. As the number of DDN layers
increases, the representational space of the outputs expands exponentially, and
the generated samples become increasingly similar to the GT. This hierarchical
output pattern of discrete distributions endows DDN with two intriguing
properties: highly compressed representation and more general zero-shot
conditional generation. We demonstrate the efficacy of DDN and these intriguing
properties through experiments on CIFAR-10 and FFHQ.

通过使用层次离散分布生成多个离散样本点，逐渐拟合目标分布，这种离散分布的层次输出模式赋予了 DDN 两个有趣的特性：高度压缩表示和更一般的零样本条件生成。我们通过在 CIFAR-10 和 FFHQ 上进行实验验证了 DDN 及其有趣特性的有效性。