Epistemic Uncertainty is a measure of the lack of knowledge of a learner
which diminishes with more evidence. While existing work focuses on using the
variance of the Bayesian posterior due to parameter uncertainty as a measure of
epistemic uncertainty, we argue that this does not capture the part of lack of
knowledge induced by model misspecification. We discuss how the excess risk,
which is the gap between the generalization error of a predictor and the Bayes
predictor, is a sound measure of epistemic uncertainty which captures the
effect of model misspecification. We thus propose a principled framework for
directly estimating the excess risk by learning a secondary predictor for the
generalization error and subtracting an estimate of aleatoric uncertainty,
i.e., intrinsic unpredictability. We discuss the merits of this novel measure
of epistemic uncertainty, and highlight how it differs from variance-based
measures of epistemic uncertainty and addresses its major pitfall. Our
framework, Direct Epistemic Uncertainty Prediction (DEUP) is particularly
interesting in interactive learning environments, where the learner is allowed
to acquire novel examples in each round. Through a wide set of experiments, we
illustrate how existing methods in sequential model optimization can be
improved with epistemic uncertainty estimates from DEUP, and how DEUP can be
used to drive exploration in reinforcement learning. We also evaluate the
quality of uncertainty estimates from DEUP for probabilistic image
classification and predicting synergies of drug combinations.

本文提出了一种直接估计超额风险的基于学习次级预测器和减去自身不确定性估计的框架，探讨了其与贝叶斯后验方差等主流方法的不同和改进点，并在一系列实验中展示了这种新方法在迭代学习和探索性强化学习等方面的优越性。