Gaussian graphical models (GGMs) are widely used for statistical modeling,
because of ease of inference and the ubiquitous use of the normal distribution
in practical approximations. However, they are also known for their limited
modeling abilities, due to the Gaussian assumption. In this paper, we introduce
a novel variant of GGMs, which relaxes the Gaussian restriction and yet admits
efficient inference. Specifically, we impose a bipartite structure on the GGM
and govern the hidden variables by truncated normal distributions. The
nonlinearity of the model is revealed by its connection to rectified linear
unit (ReLU) neural networks. Meanwhile, thanks to the bipartite structure and
appealing properties of truncated normals, we are able to train the models
efficiently using contrastive divergence. We consider three output constructs,
accounting for real-valued, binary and count data. We further extend the model
to deep constructions and show that deep models can be used for unsupervised
pre-training of rectifier neural networks. Extensive experimental results are
provided to validate the proposed models and demonstrate their superiority over
competing models.

本文介绍了一种新型 GGMs 变体，其通过在 GGM 上施加二分结构，并使用截断正态分布来管理隐藏变量以放宽高斯限制，实现了非常有效的推断，进而建立了与修正线性单元（ReLU）神经网络的连接，用于真实值，二进制和计数数据的输出构建，还将该模型扩展到深层结构，并展示了深层模型可以用于修正单元神经网络的无监督预训练。大量实验证据证明了提出的模型相对于竞争模型的优越性。