The Neural Autoregressive Distribution Estimator (NADE) and its real-valued version RNADE are competitive density models of multidimensional data across a variety of domains. These models use a fixed, arbitrary ordering of the data dimensions. One can easily condition on variables at the beginning of the ordering, and marginalize out variables at the end of the ordering, however other inference tasks require approximate inference. In this work we introduce an efficient procedure to simultaneously train a NADE model for each possible ordering of the variables, by sharing parameters across all these models. We can thus use the most convenient model for each inference task at hand, and ensembles of such models with different orderings are immediately available. Moreover, unlike the original NADE, our training procedure scales to deep models. Empirically, ensembles of Deep NADE models obtain state of the art density estimation performance.

本文介绍一种新的方法，即同时为每个变量的可能排序训练NADE模型，共享跨所有模型的参数。这种方法既可以为不同推理任务使用最方便的模型，又可以马上使用具有不同排序的模型集成。此外，我们的训练方法可以扩展到深度模型，实验证明深度NADE模型的集合可以获得最先进的密度估计性能。

一种深度可计算密度估计器