A lot of effort is currently invested in safeguarding autonomous driving
systems, which heavily rely on deep neural networks for computer vision. We
investigate the coupling of different neural network calibration measures with
a special focus on the Area Under the Sparsification Error curve (AUSE) metric.
We elaborate on the well-known inconsistency in determining optimal calibration
using the Expected Calibration Error (ECE) and we demonstrate similar issues
for the AUSE, the Uncertainty Calibration Score (UCS), as well as the
Uncertainty Calibration Error (UCE). We conclude that the current methodologies
leave a degree of freedom, which prevents a unique model calibration for the
homologation of safety-critical functionalities. Furthermore, we propose the
AUSE as an indirect measure for the residual uncertainty, which is irreducible
for a fixed network architecture and is driven by the stochasticity in the
underlying data generation process (aleatoric contribution) as well as the
limitation in the hypothesis space (epistemic contribution).

当前大量的工作专注于保护自主驾驶系统，这些系统在计算机视觉方面严重依赖于深度神经网络。我们研究了不同神经网络校准度量的耦合，重点关注稀疏化误差曲线下的面积（AUSE）指标。我们详述了确定最佳校准度使用期望校准误差（ECE）时存在的众所周知的一致性问题，并且我们还展示了 AUSE、不确定性校准分数（UCS）以及不确定性校准误差（UCE）也存在类似问题。我们得出结论，目前的方法留下了一定的自由度，这使得无法为安全关键功能的鉴定提供独特的模型校准。此外，我们提出 AUSE 作为间接度量残余不确定性的方法，而这种不确定性对于固定的网络架构是不可减少的，这是由于底层数据生成过程中的随机性贡献以及假设空间限制所驱动的。