It is widely recognized that the generalization ability of neural networks
can be greatly enhanced through carefully designing the training procedure. The
current state-of-the-art training approach involves utilizing stochastic
gradient descent (SGD) or Adam optimization algorithms along with a combination
of additional regularization techniques such as weight decay, dropout, or noise
injection. Optimal generalization can only be achieved by tuning a multitude of
hyperparameters through grid search, which can be time-consuming and
necessitates additional validation datasets. To address this issue, we
introduce a practical PAC-Bayes training framework that is nearly tuning-free
and requires no additional regularization while achieving comparable testing
performance to that of SGD/Adam after a complete grid search and with extra
regularizations. Our proposed algorithm demonstrates the remarkable potential
of PAC training to achieve state-of-the-art performance on deep neural networks
with enhanced robustness and interpretability.

提出一种实用的 PAC-Bayes 训练框架，通过使用无需多重超参数调整的算法，结合 Stochastic Gradient Descent（SGD）或 Adam 优化算法和正则化技术等方法，实现了与常规方法相当的测试性能，同时实现了深度神经网络的鲁棒性和可解释性。