In safety-critical classification tasks, conformal prediction allows to
perform rigorous uncertainty quantification by providing confidence sets
including the true class with a user-specified probability. This generally
assumes the availability of a held-out calibration set with access to ground
truth labels. Unfortunately, in many domains, such labels are difficult to
obtain and usually approximated by aggregating expert opinions. In fact, this
holds true for almost all datasets, including well-known ones such as CIFAR and
ImageNet. Applying conformal prediction using such labels underestimates
uncertainty. Indeed, when expert opinions are not resolvable, there is inherent
ambiguity present in the labels. That is, we do not have ``crisp'', definitive
ground truth labels and this uncertainty should be taken into account during
calibration. In this paper, we develop a conformal prediction framework for
such ambiguous ground truth settings which relies on an approximation of the
underlying posterior distribution of labels given inputs. We demonstrate our
methodology on synthetic and real datasets, including a case study of skin
condition classification in dermatology.

在安全关键的分类任务中，我们提出了一种适用于含有模糊标签的情况的 conformal prediction 框架，在 approximated 的标签的基础上通过近似输入的后验分布来进行不确定性的校准。我们在合成和真实数据集上验证了我们的方法，并在皮肤病学中患者条件分类的案例研究中进行了实证。