In this paper, we study sampling from a posterior derived from a neural network. We propose a new probabilistic model consisting of adding noise at every pre- and post-activation in the network, arguing that the resulting posterior can be sampled using an efficient Gibbs sampler. The Gibbs sampler attains similar performances as the state-of-the-art Monte Carlo Markov chain methods, such as the Hamiltonian Monte Carlo or the Metropolis adjusted Langevin algorithm, both on real and synthetic data. By framing our analysis in the teacher-student setting, we introduce a thermalization criterion that allows us to detect when an algorithm, when run on data with synthetic labels, fails to sample from the posterior. The criterion is based on the fact that in the teacher-student setting we can initialize an algorithm directly at equilibrium.

通过在神经网络各个节点引入噪声，作者提出一种新的概率模型，认为由此所得的 posterior distribution 可以用有效的 Gibbs sampler 来采样，并在实验中证明了该方法在真实数据和合成数据上都能取得类似于 Hamiltonian Monte Carlo 或 Metropolis adjusted Langevin algorithm 等蒙特卡罗马尔可夫链方法的性能；此外，通过教师-学生模型，作者提出了一种热化准则，并通过实验演示其有效性，该准则可以用来检测算法在合成标注数据上无法从 posterior distribution 中采样的情况。

神经网络后验的吉布斯采样