The key distinguishing property of a Bayesian approach is marginalization
instead of optimization, not the prior, or Bayes rule. Bayesian inference is
especially compelling for deep neural networks. (1) Neural networks are
typically underspecified by the data, and can represent many different but high
performing models corresponding to different settings of parameters, which is
exactly when marginalization will make the biggest difference for both
calibration and accuracy. (2) Deep ensembles have been mistaken as competing
approaches to Bayesian methods, but can be seen as approximate Bayesian
marginalization. (3) The structure of neural networks gives rise to a
structured prior in function space, which reflects the inductive biases of
neural networks that help them generalize. (4) The observed correlation between
parameters in flat regions of the loss and a diversity of solutions that
provide good generalization is further conducive to Bayesian marginalization,
as flat regions occupy a large volume in a high dimensional space, and each
different solution will make a good contribution to a Bayesian model average.
(5) Recent practical advances for Bayesian deep learning provide improvements
in accuracy and calibration compared to standard training, while retaining
scalability.

贝叶斯方法在深度神经网络中应用广泛，因为它通过边缘化取代优化，能够提高模型的校准性和准确性，并利用神经网络结构中的先验偏好来帮助泛化。此外，最近的贝叶斯深度学习实践进展也提高了模型的准确性、校准性和可伸缩性。