Recent work has observed that one can outperform exact inference in Bayesian neural networks by tuning the "temperature" of the posterior on a validation set (the "cold posterior" effect). To help interpret this phenomenon, we argue that commonly used priors in Bayesian neural networks can significantly overestimate the aleatoric uncertainty in the labels on many classification datasets. This problem is particularly pronounced in academic benchmarks like MNIST or CIFAR, for which the quality of the labels is high. For the special case of Gaussian process regression, any positive temperature corresponds to a valid posterior under a modified prior, and tuning this temperature is directly analogous to empirical Bayes. On classification tasks, there is no direct equivalence between modifying the prior and tuning the temperature, however reducing the temperature can lead to models which better reflect our belief that one gains little information by relabeling existing examples in the training set. Therefore although cold posteriors do not always correspond to an exact inference procedure, we believe they may often better reflect our true prior beliefs.

本文探讨了后验温度的调整在贝叶斯神经网络中的作用以及先验的影响，研究发现，在高质量的分类数据集如MNIST或CIFAR中，经常使用的先验能够显著地高估标签的不确定性。研究在高斯过程回归中发现，任何正数的温度都对应于一个经过修改的先验下的有效后验，而调整后验温度直接类比于经验贝叶斯。当应用于分类任务时，调整先验并不直接等同于调整后验温度，但是降低后验温度可以得到更好地反映新增训练样本信息的模型。因此，虽然冷后验并不总是对应于精确的推理过程，但我们认为，它们可能更好地反映了我们真实的先验信念。

冷后验分布与随机不确定性