Classical variational inference for Bayesian neural networks (BNNs) in parameter space usually suffers from unresolved prior issues such as knowledge encoding intractability and pathological behaviors in deep networks, which could lead to an improper posterior inference. Hence, functional variational inference has been proposed recently to resolve these issues via stochastic process priors. Beyond variational inference, stochastic gradient Markov Chain Monte Carlo (SGMCMC) is another scalable and effective inference method for BNNs to asymptotically generate samples from true posterior by simulating a continuous dynamic. However, the existing SGMCMC methods only work in parametric space, which has the same issues of parameter-space variational inference, and extending the parameter-space dynamics to function-space dynamics is not a trivial undertaking. In this paper, we introduce a new functional SGMCMC scheme via newly designed diffusion dynamics, which can incorporate more informative functional priors. Moreover, we prove that the stationary distribution of these functional dynamics is the target posterior distribution over functions. We demonstrate better performance in both accuracy and uncertainty quantification of our functional SGMCMC on several tasks compared with naive SGMCMC and functional variational inference methods.

本研究解决了贝叶斯神经网络中经典变分推断在参数空间面临的先验问题，通过提出一种新的功能随机梯度马尔可夫链蒙特卡洛（SGMCMC）方案，能够在功能空间中生成更真实的后验样本。研究结果表明，该方法在准确性和不确定性量化方面优于现有的SGMCMC和功能变分推断方法，具有显著的应用潜力。

基于功能的随机梯度马尔可夫链蒙特卡洛方法在贝叶斯神经网络中的应用