Bayesian methods hold significant promise for improving the uncertainty quantification ability and robustness of deep neural network models. Recent research has seen the investigation of a number of approximate Bayesian inference methods for deep neural networks, building on both the variational Bayesian and Markov chain Monte Carlo (MCMC) frameworks. A fundamental issue with MCMC methods is that the improvements they enable are obtained at the expense of increased computation time and model storage costs. In this paper, we investigate the potential of sparse network structures to flexibly trade-off model storage costs and inference run time against predictive performance and uncertainty quantification ability. We use stochastic gradient MCMC methods as the core Bayesian inference method and consider a variety of approaches for selecting sparse network structures. Surprisingly, our results show that certain classes of randomly selected substructures can perform as well as substructures derived from state-of-the-art iterative pruning methods while drastically reducing model training times.

本文探讨使用随机梯度马尔科夫链蒙特卡罗方法作为Bayesian推理的核心方法，考虑各种选择稀疏网络结构的方法，以灵活地权衡模型存储成本和推理运行时间与预测性能和不确定性量化能力之间的潜力。实验结果表明，某些随机选择的子结构可以像从最先进的迭代剪枝方法中获得的子结构一样表现得很好，同时大大缩短模型训练时间。

参数稀疏性对于贝叶斯深度学习中随机梯度MCMC方法的影响