To get Bayesian neural networks to perform comparably to standard neural
networks it is usually necessary to artificially reduce uncertainty using a
"tempered" or "cold" posterior. This is extremely concerning: if the prior is
accurate, Bayes inference/decision theory is optimal, and any artificial
changes to the posterior should harm performance. While this suggests that the
prior may be at fault, here we argue that in fact, BNNs for image
classification use the wrong likelihood. In particular, standard image
benchmark datasets such as CIFAR-10 are carefully curated. We develop a
generative model describing curation which gives a principled Bayesian account
of cold posteriors, because the likelihood under this new generative model
closely matches the tempered likelihoods used in past work.

本文讨论贝叶斯神经网络在图片分类上的应用，发现此类应用使用的是错误的似然度。作者开发了一个描述 “筛选” 过程的生成模型，并与之前使用的调整后的后验概率似然度进行了基于贝叶斯思想的对比。

深度神经网络中冷后验的统计理论

A statistical theory of cold posteriors in deep neural networks

During the past five years the Bayesian deep learning community has developed
increasingly accurate and efficient approximate inference procedures that allow
for Bayesian inference in deep neural networks. However, despite this
algorithmic progress and the promise of improved uncertainty quantification and
sample efficiency there are---as of early 2020---no publicized deployments of
Bayesian neural networks in industrial practice. In this work we cast doubt on
the current understanding of Bayes posteriors in popular deep neural networks:
we demonstrate through careful MCMC sampling that the posterior predictive
induced by the Bayes posterior yields systematically worse predictions compared
to simpler methods including point estimates obtained from SGD. Furthermore, we
demonstrate that predictive performance is improved significantly through the
use of a "cold posterior" that overcounts evidence. Such cold posteriors
sharply deviate from the Bayesian paradigm but are commonly used as heuristic
in Bayesian deep learning papers. We put forward several hypotheses that could
explain cold posteriors and evaluate the hypotheses through experiments. Our
work questions the goal of accurate posterior approximations in Bayesian deep
learning: If the true Bayes posterior is poor, what is the use of more accurate
approximations? Instead, we argue that it is timely to focus on understanding
the origin of the improved performance of cold posteriors.

本文研究贝叶斯神经网络中后验概率的真实性及精度，通过 MCMC 抽样实验发现使用贝叶斯后验的预测性能显著不如基于 SGD 的点估计方法，并提出了一种 “冷后验” 的新方法，解释其使用背景及理论基础。本工作挑战了贝叶斯深度学习中关于精度估算的目标，提出从理解和探究 “冷后验” 方法优化神经网络性能入手。