While machine learning models capable of producing uncertainty estimates are becoming widespread, uncalibrated uncertainty estimates are often overconfident, and often assume predetermined probability distributions over the error which do not match the empirical calibration error. Most work on calibrating uncertainty estimates focuses on classification rather than regression, which introduces novel challenges. We present a calibration method referred to as Calibrating Regression Uncertainty Distributions Empirically (CRUDE) that does not assume a fixed uncertainty distribution, instead making the weaker assumption that error distributions have a constant shape across the output space, shifted by predicted mean and scaled by predicted standard deviation. CRUDE requires no training of the calibration estimator aside from a parameter to account for consistent bias in the predicted mean. Our method is distribution-agnostic and provides sharper and more accurate uncertainty estimates than state of the art techniques, as demonstrated by calibration and sharpness measures across many datasets.

该研究提出了一种用于回归设置的校准方法，称为经验性地校准回归不确定性分布（CRUDE），该方法不假定错误的特定不确定性分布，并展示了它的理论联系以及 CRUDE 的性能优于现有技术。

经验法调整回归不确定性分布