We introduce a variational Bayesian neural network where the parameters are governed via a probability distribution on \emph{random matrices}. Specifically, we employ a matrix variate Gaussian~\cite{gupta1999matrix} parameter posterior distribution where we explicitly model the covariance among the input and output dimensions of each layer. Furthermore, with approximate covariance matrices we can achieve a more efficient way to represent those correlations that is also cheaper than fully factorized parameter posteriors. We further show that with the "local reprarametrization trick"~\cite{kingma2015variational} on this posterior distribution we arrive at a Gaussian Process~\cite{rasmussen2006gaussian} interpretation of the hidden units in each layer and we, similarly with~\cite{gal2015dropout}, provide connections with deep Gaussian processes. We continue in taking advantage of this duality and incorporate "pseudo-data"~\cite{snelson2005sparse} in our model, which in turn allows for more efficient sampling while maintaining the properties of the original model. The validity of the proposed approach is verified through extensive experiments.

本文介绍了一种使用概率分布的随机矩阵来管理参数的变分贝叶斯神经网络，并使用矩阵变量高斯参数后验分布来明确建模每个层的输入和输出维度之间的协方差。此外，使用近似协方差矩阵，可以实现比完全分解更高效且更便宜的表示，同时无需损失模型性能。通过引入“局部重参数化技巧”，可以将此后验分布转换为高斯过程，从而为每个层的隐藏单元提供解释，与深度高斯过程建立联系，并结合伪数据提高了模型采样效率。实验表明所提出的方法的有效性。

基于矩阵高斯后验的结构化高效变分深度学习