We introduce a principled approach for unsupervised structure learning of deep neural networks. We propose a new interpretation for depth and inter-layer connectivity where conditional independencies in the input distribution are encoded hierarchically in the network structure. Thus, the depth of the network is determined inherently (equal to the maximal order of independence in the input distribution). The proposed method casts the problem of neural network structure learning as a problem of Bayesian network structure learning. Then, instead of directly learning the discriminative structure, it learns a generative graph, constructs its stochastic inverse, and then constructs a discriminative graph. We prove that conditional-dependency relations among the latent variables in the generative graph are preserved in the class-conditional discriminative graph. We demonstrate on image classification benchmarks that the deepest layers (convolutional and dense) of common networks can be replaced by significantly smaller learned structures, while maintaining classification accuracy---state-of-the-art on tested benchmarks. Our structure learning algorithm requires a small computational cost and runs efficiently on a standard desktop CPU.

本文提出一种基于贝叶斯网络结构学习的方法，用于无监督结构学习深度神经网络，通过生成图，构建其随机反向，然后构建一个判别图，证明生成图的潜变量之间的条件依赖关系在条件“分类条件下”丢失在判别图，从而实现通用网络深层（卷积和密集）的学习结构替代，在保持分类准确性的同时显著减少计算成本。

通过贝叶斯网络结构学习构建深度神经网络