Traditional deep learning (DL) models are powerful classifiers, but many
approaches do not provide uncertainties for their estimates. Uncertainty
quantification (UQ) methods for DL models have received increased attention in
the literature due to their usefulness in decision making, particularly for
high-consequence decisions. However, there has been little research done on how
to evaluate the quality of such methods. We use statistical methods of
frequentist interval coverage and interval width to evaluate the quality of
credible intervals, and expected calibration error to evaluate classification
predicted confidence. These metrics are evaluated on Bayesian neural networks
(BNN) fit using Markov Chain Monte Carlo (MCMC) and variational inference (VI),
bootstrapped neural networks (NN), Deep Ensembles (DE), and Monte Carlo (MC)
dropout. We apply these different UQ for DL methods to a hyperspectral image
target detection problem and show the inconsistency of the different methods'
results and the necessity of a UQ quality metric. To reconcile these
differences and choose a UQ method that appropriately quantifies the
uncertainty, we create a simulated data set with fully parameterized
probability distribution for a two-class classification problem. The gold
standard MCMC performs the best overall, and the bootstrapped NN is a close
second, requiring the same computational expense as DE. Through this
comparison, we demonstrate that, for a given data set, different models can
produce uncertainty estimates of markedly different quality. This in turn
points to a great need for principled assessment methods of UQ quality in DL
applications.

对深度学习的 UQ 方法进行了质量评估，比较了不同模型的不确定性估计质量。

神经网络不确定性估计在分类问题中的质量比较

Comparing the quality of neural network uncertainty estimates for  classification problems

Contrastively trained encoders have recently been proven to invert the
data-generating process: they encode each input, e.g., an image, into the true
latent vector that generated the image (Zimmermann et al., 2021). However,
real-world observations often have inherent ambiguities. For instance, images
may be blurred or only show a 2D view of a 3D object, so multiple latents could
have generated them. This makes the true posterior for the latent vector
probabilistic with heteroscedastic uncertainty. In this setup, we extend the
common InfoNCE objective and encoders to predict latent distributions instead
of points. We prove that these distributions recover the correct posteriors of
the data-generating process, including its level of aleatoric uncertainty, up
to a rotation of the latent space. In addition to providing calibrated
uncertainty estimates, these posteriors allow the computation of credible
intervals in image retrieval. They comprise images with the same latent as a
given query, subject to its uncertainty.

该研究提出了一种基于分布预测的信息最大化方法，用于比较训练编码器的逆向重建任务，并且可以根据图像检索任务的不确定性估计出可信区间.