The large-scale pre-trained neural network has achieved notable success in
enhancing performance for downstream tasks. Another promising approach for
generalization is Bayesian Neural Network (BNN), which integrates Bayesian
methods into neural network architectures, offering advantages such as Bayesian
Model averaging (BMA) and uncertainty quantification. Despite these benefits,
transfer learning for BNNs has not been widely investigated and shows limited
improvement. We hypothesize that this issue arises from the inability to find
flat minima, which is crucial for generalization performance. To address this,
we evaluate the sharpness of BNNs in various settings, revealing their
insufficiency in seeking flat minima and the influence of flatness on BMA
performance. Therefore, we propose Sharpness-aware Bayesian Model Averaging
(SA-BMA), a Bayesian-fitting flat posterior seeking optimizer integrated with
Bayesian transfer learning. SA-BMA calculates the divergence between posteriors
in the parameter space, aligning with the nature of BNNs, and serves as a
generalized version of existing sharpness-aware optimizers. We validate that
SA-BMA improves generalization performance in few-shot classification and
distribution shift scenarios by ensuring flatness.

我们提出了一种基于概率神经网络和贝叶斯传递学习的锐度感知贝叶斯模型平均方法，通过寻找平的极小值来提高模型的泛化性能。