There has been a lot of recent interest in designing neural network models to estimate a distribution from a set of examples. We introduce a simple modification for autoencoder neural networks that yields powerful generative models. Our method masks the autoencoder's parameters to respect autoregressive constraints: each input is reconstructed only from previous inputs in a given ordering. Constrained this way, the autoencoder outputs can be interpreted as a set of conditional probabilities, and their product, the full joint probability. We can also train a single network that can decompose the joint probability in multiple different orderings. Our simple framework can be applied to multiple architectures, including deep ones. Vectorized implementations, such as on GPUs, are simple and fast. Experiments demonstrate that this approach is competitive with state-of-the-art tractable distribution estimators. At test time, the method is significantly faster and scales better than other autoregressive estimators.

本研究提出了一种简单的神经网络自编码器模型，可以遵守自回归约束实现强大的生成模型，其输出结果可以被理解为一组条件概率，可以用于多种架构，包括深度架构，并通过实验证明其与现有的可计算分布估算器的表现相当竞争力，并且测试时速度显著更快，规模更大。

MADE: 遮蔽自编码器用于分布估计