Energy-based models (EBM) have become increasingly popular within computer
vision. EBMs bring a probabilistic approach to training deep neural networks
(DNN) and have been shown to enhance performance in areas such as calibration,
out-of-distribution detection, and adversarial resistance. However, these
advantages come at the cost of estimating input data probabilities, usually
using a Langevin based method such as Stochastic Gradient Langevin Dynamics
(SGLD), which bring additional computational costs, require parameterization,
caching methods for efficiency, and can run into stability and scaling issues.
EBMs use dynamical methods to draw samples from the probability density
function (PDF) defined by the current state of the network and compare them to
the training data using a maximum log likelihood approach to learn the correct
PDF.
We propose a non-generative training approach, Non-Generative EBM (NG-EBM),
that utilizes the {\it{Approximate Mass}}, identified by Grathwohl et al., as a
loss term to direct the training. We show that our NG-EBM training strategy
retains many of the benefits of EBM in calibration, out-of-distribution
detection, and adversarial resistance, but without the computational complexity
and overhead of the traditional approaches. In particular, the NG-EBM approach
improves the Expected Calibration Error by a factor of 2.5 for CIFAR10 and 7.5
times for CIFAR100, when compared to traditionally trained models.

本文提出 Non-Generative EBM 方法作为一种有效的训练方法，可以在保持 EBM 中关键的稳定性和性能的同时，降低计算复杂度和开销。该方法能够大幅提高 CIFAR10 和 CIFAR100 数据集的预期校准误差。