Uncertainty quantification, once a singular task, has evolved into a spectrum
of tasks, including abstained prediction, out-of-distribution detection, and
aleatoric uncertainty quantification. The latest goal is disentanglement: the
construction of multiple estimators that are each tailored to one and only one
task. Hence, there is a plethora of recent advances with different intentions -
that often entirely deviate from practical behavior. This paper conducts a
comprehensive evaluation of numerous uncertainty estimators across diverse
tasks on ImageNet. We find that, despite promising theoretical endeavors,
disentanglement is not yet achieved in practice. Additionally, we reveal which
uncertainty estimators excel at which specific tasks, providing insights for
practitioners and guiding future research toward task-centric and disentangled
uncertainty estimation methods. Our code is available at
this https URL

通过对 ImageNet 上的多个任务进行综合评估，本文发现尽管有许多有前景的理论努力，实际上还没有实现解缠，同时揭示了哪些不确定性估计器在特定任务上表现出色，为从业者提供了见解，并指导未来研究朝向以任务为中心和解缠的不确定性估计方法。