During the past five years the Bayesian deep learning community has developed
increasingly accurate and efficient approximate inference procedures that allow
for Bayesian inference in deep neural networks. However, despite this
algorithmic progress and the promise of improved uncertainty quantification and
sample efficiency there are---as of early 2020---no publicized deployments of
Bayesian neural networks in industrial practice. In this work we cast doubt on
the current understanding of Bayes posteriors in popular deep neural networks:
we demonstrate through careful MCMC sampling that the posterior predictive
induced by the Bayes posterior yields systematically worse predictions compared
to simpler methods including point estimates obtained from SGD. Furthermore, we
demonstrate that predictive performance is improved significantly through the
use of a "cold posterior" that overcounts evidence. Such cold posteriors
sharply deviate from the Bayesian paradigm but are commonly used as heuristic
in Bayesian deep learning papers. We put forward several hypotheses that could
explain cold posteriors and evaluate the hypotheses through experiments. Our
work questions the goal of accurate posterior approximations in Bayesian deep
learning: If the true Bayes posterior is poor, what is the use of more accurate
approximations? Instead, we argue that it is timely to focus on understanding
the origin of the improved performance of cold posteriors.

本文研究贝叶斯神经网络中后验概率的真实性及精度，通过 MCMC 抽样实验发现使用贝叶斯后验的预测性能显著不如基于 SGD 的点估计方法，并提出了一种 “冷后验” 的新方法，解释其使用背景及理论基础。本工作挑战了贝叶斯深度学习中关于精度估算的目标，提出从理解和探究 “冷后验” 方法优化神经网络性能入手。