In this work, we advocate for the importance of singular learning theory (SLT) as it pertains to the theory and practice of variational inference in Bayesian neural networks (BNNs). To begin, using SLT, we lay to rest some of the confusion surrounding discrepancies between downstream predictive performance measured via e.g., the test log predictive density, and the variational objective. Next, we use the SLT-corrected asymptotic form for singular posterior distributions to inform the design of the variational family itself. Specifically, we build upon the idealized variational family introduced in \citet{bhattacharya_evidence_2020} which is theoretically appealing but practically intractable. Our proposal takes shape as a normalizing flow where the base distribution is a carefully-initialized generalized gamma. We conduct experiments comparing this to the canonical Gaussian base distribution and show improvements in terms of variational free energy and variational generalization error.

本文研究奉行奇异学习理论（SLT）在变分推断中的重要性，使用 SLT 填补变分推断与下游预测性能之间差异的众多困惑，使用 SLT 修正奇异后验分布的渐近形式以支持变分族设计，并提出了基于谨慎初始化的广义 Gamma 分布的归一化流作为改进方案，相关实验也证实改进的可行性和效果。

奇点分辨解法的变分贝叶斯神经网络