Artificial neural networks (ANNs) are powerful machine learning methods used
in many modern applications such as facial recognition, machine translation,
and cancer diagnostics. A common issue with ANNs is that they usually have
millions or billions of trainable parameters, and therefore tend to overfit to
the training data. This is especially problematic in applications where it is
important to have reliable uncertainty estimates. Bayesian neural networks
(BNN) can improve on this, since they incorporate parameter uncertainty. In
addition, latent binary Bayesian neural networks (LBBNN) also take into account
structural uncertainty by allowing the weights to be turned on or off, enabling
inference in the joint space of weights and structures. In this paper, we will
consider two extensions to the LBBNN method: Firstly, by using the local
reparametrization trick (LRT) to sample the hidden units directly, we get a
more computationally efficient algorithm. More importantly, by using
normalizing flows on the variational posterior distribution of the LBBNN
parameters, the network learns a more flexible variational posterior
distribution than the mean field Gaussian. Experimental results show that this
improves predictive power compared to the LBBNN method, while also obtaining
more sparse networks. We perform two simulation studies. In the first study, we
consider variable selection in a logistic regression setting, where the more
flexible variational distribution leads to improved results. In the second
study, we compare predictive uncertainty based on data generated from
two-dimensional Gaussian distributions. Here, we argue that our Bayesian
methods lead to more realistic estimates of predictive uncertainty.

本文介绍了使用本地重新参数化技巧（LRT）和归一化流在 LBBNN 方法的变分后验分布上的应用，以改进预测性能并获得更稀疏的网络，通过两个模拟研究证明这些贝叶斯方法的使用可以得到更为实际的预测不确定性估计。