Bayesian inference for neural networks, or Bayesian deep learning, has the potential to provide well-calibrated predictions with quantified uncertainty and robustness. However, the main hurdle for Bayesian deep learning is its computational complexity due to the high dimensionality of the parameter space. In this work, we propose a novel scheme that addresses this limitation by constructing a low-dimensional subspace of the neural network parameters-referred to as an active subspace-by identifying the parameter directions that have the most significant influence on the output of the neural network. We demonstrate that the significantly reduced active subspace enables effective and scalable Bayesian inference via either Monte Carlo (MC) sampling methods, otherwise computationally intractable, or variational inference. Empirically, our approach provides reliable predictions with robust uncertainty estimates for various regression tasks.

本研究提出了一种新颖的方案，通过构建神经网络参数的低维子空间（称为活跃子空间）来解决贝叶斯深度学习中由于参数空间的高维导致的计算复杂性限制。我们展示了显著降低的活跃子空间，通过Monte Carlo（MC）采样方法或变分推断实现了可行和可扩展的贝叶斯推断，为各种回归任务提供了可靠的预测和鲁棒的不确定性估计。

深度神经网络中的有效可扩展不确定性量化学习主动子空间